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ABSTRACT 




The .purpose of this praMMim was to develop and tmplement 
a tGniputerlzed monitoring system fbr Math 1 1* grades ^, 5, and 6» 
(The samet system, once estabi Ished, can be u^ed.f^r monitoring 
Readlhg as Well) 1^1 three yeahs. The hypothesis (s that such an 
Imprfe^entatton will Improve student performance both on norm- 
referertted and cr I terlon- referenced tests^ The practlcum was ' 
successfully executed and the hypothesis was confirmed, ' , 
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INTRODUCTION . • 

this report win detail a three year labor'- for the purpose 
of helping students perform better academical lyl The Needs wll/ 
discuss th^ context prior to the establishment of this jponltOrtng . 
system. Section Kli the Problem will detail the confl let between 

• . ^ ■■ > ■ , - ' : i'' ' •• ■■ 

norm-referenced and criterion-referenced forms of education. The 

■ ■ ' ■ ■. . " ■ '/ ■ " " ' . 

different* views of What constitutes learning should be made evident - 

Section III will" describe the Brentwood, New York appHcatjon of 

Comprehensive Achievement Monitoring., The procedures', forms and 

systems will be explained. In Section IV there is a discussion 

and explanation of the Inpteriial that needed". to be developjed to 

I ' - . ■■ . ■ i . 

help parents, students and 'teachers Implement and Interpret the 
system. Section V wil] detail the sequences of activities^ re- 
quired for^use of the monitor lug system. Section VI will deal 
primarily with the outcomes 'bf the pr^qess.; that Is, the use of 
the Information to mbdlfy behavioK materials and course content 
to he 1 1f) students improve performance. Section VI T reports On the 
back-^up system jrtThlch provlies weta Wed Information ori each objective. 
.ln> Sect Ion Vl 1 1 the course qf Implementation of this program In the 

Brentwood Schools is narrated. In Section IX the results in terms 

- ^ . ' . ^ »^ 

of student per'formance are repoi^ted. Section X discusses, the budget 

•. . • • ^ ■ 

x. . . \ - 

from the beginning of the program to the present date. 
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. X» Needs. Brentwood Is a commjjnlty In the middle of tong Island^ 

' ■ - ■ > ' "i^. ' . ' . • * 
Pri<)r"to 195p;^it was spars^ly^settled. The eixtension pf the Long . - \ 




■ . 1 ' 




'. . . ■ ■ 
.ft 

• • 

1 

' • . . 1 

' .- - 


Istand Rail r-oad plus the population explosion bi^ th6 T^SO's aM . 
1960's brought >^pld development. By 1970; there were more, than 
^60,060 people In the. community and 22,000' studen^ts* in grades Krl2. ^^'^T^ 
Twenty large schools had been built and yet the secondary schools * 

were on split-session. ^Thte population of the school distViotV It 

". • ' * ■ * , . ' -. - • ^ ^ 

was reported to the state, was 13^ Spanish $urnamed^- S% black and 


.']■'■■ 

■ * • < 




82^ White. The ^tommuni ty devefl^ed as, housing projects which were 
occupied as quickly as they cpuld be bullf. Some V/orkers commute 




i - '■ 






, i> ' ■ . " - ■ . • . ^-^ . ■ - , ' , ^ ■ * 
, to the* city*, nearly fifty miles awdy. However, most commute to*otlj^r 


*• I 




communlties'TUUth closer. Brentwood has not developed extensive in-* 


* ■ "•*) 




dustry of *ft<; own. * • , 






^ Student performance on ^^tarjdard I zed tests Indicated a narrow 






top band of achievement and wide band of sub norm' performance. In 






1970, on Ca'l Ifornia Acbievfement tesfts and New York St^te Tests, It . - 
seemed clear that the |:rend nad deveTopedw to a critical state. 
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;* 1970 PEP Tests . ' ' ^' 
% Below Minimum Competence 


' ■ j 




> Grade Reading t Math 


1 




. 3 ■ ... 2k .19 ./ ' \ 


n 




v" 6 • 31 32 






The educatlpnal -fundamentals of math and reading were noL being sue-* 
icessful ly taught . Success, was not possible In our secondary schools 
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. .V, . ■ "^i ' — ; ^ — ' ; — 

^- ■ • . ' ■ -2-^ •■■ ^ ■ v. .• :.. 

The Asslstanr Saperlntendefit apfl th!s wrfter met extensively 




. ittt develop a.!5„trategy for reversing the| dlsasterous trends revealed 
iiy the norm r^fer^nced test Scores. M . ; / • > 






V A buddet reject Ions, had forced a 






\ duplet f on of the supervision staff In the Brdntwood Schools., Where 




■ ■ ^ . •- 


there had once been a full time Mathen)atic5 Coordinator and staff 




" **■ 


Vand ,^ full time R^adlnd Coordinator ahd staff, there were now none 
left/ An outcome was t^^ this period there ,wa? a decreasing 


* • 




aimpunt of supervision and mon the mathematics and 
,rea'Jlng programs. ; It was very clear, frolTi.recen^^ 




■ ♦ 


there was no chance of hiring personnel to conduct supervision of 






the program. As 'we studied our .problem and consulted other school 






■ ' . " . • i ■ . ° A - . 

V districts, |he New York State ^ducatl^n Department and local unlver- 

' '. •„ . ' ^ . " ' A , • ■' ■ * " 




ft ■ , , ■ ^ 
' ■ -- • ' ' " 


sltl0s, another point 0f view slowly began to dominate our thinking. 




■■ / * ' ■ • 


• ; • ' \ ■ , ' ■ ■■ ■ .V . t , • . • • 
;lt seemed that new methods of njpnUtor I ng Instruction were bjeglnnln^^ 

.to evolve. . . . - *^ . 




* •■ » 


■■ ; . ' . 1 • --^ . - • 
; General Procedures of Monitoring ' 

Step T. Specify the overal 1 performance object tVes to be / 

' " * - *• - ^ ^^^^^^ . V • 




V 


accomplished through a given educational program. ^ 
The performance objectives would complete the first 

task for accouritabi 1 Ity. 

• * . ■■' ' ^ . . , . ^ 

Educational Accountability: A format for Monitoring the 
Teaching-Learning Process, Terry 0. Cornelf, £PIC Diversified 
.Systems Corporation. Educational Innovations Press, Tucson, 
- Arizona, 1971. ; > . ' 
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Step 2, Specify Ijhe time rnterval for fnonltorlng. The Instruc- 
tors that are InvoVved In the Instructional program 
shoul4 agree on the fnterval pf time that will pass 
between monitoring points^ jin most Instances., .the 
monitoring period would probably be weekly or bl- 
, weekly, " ^ 

Step 3» Selfect a person to be responsible for keeping a rec- 

» ■ - " ■ . ^ 

^.ord of each Instructor's sheets for each monitoring ^ 

• /■ • _ .■ ■ ' ' • • ■ ' - ■ 

pofnt. This person*s respdnslbl 1 1 ty would be to make 
sure that each sh^et Is completely filled out state- 
ment of objeqtfves that will be dealt with during the 
* monitoring Interval completion of Planned Program 
section at the begflnning of the monitoring Interval^ 
completion of Actual Program septlon at the end of 
the monitoring Interval , Indication of what objectives 
were met and what obJLect Ives were not met. . 

In addition to general procediiires, there are s^me Indl*-* 

• ■ ■ . • * ' ' • ^.'^ ■ • • " 

vidual Instructor procedures which should be followed If this 

V - ^ . ■ ' ■ , • ' 

nfpnitoring system Is to provide relevant feedback. The' proce- 
dures are listed Irt' sequential order. 

Individual Instructor Procedure s 
Step K Prepare interim behavioral objectives In a sequential 
order of accompl Ishment as they relate to the over- " 
all performance objectlve(s) . The, only difference 



between Interlrt objectives and the overall per- 

formance objective Is that there will be a shorter 

time Interval and the situation under which the 

. behavior will be observed will be more specific. 

« 

Step, 2. Select those object Ives that wi 1 1 be- considered for the 
' . given monitoring period. ; ^ . . * 

Step 3- Cpmplete the Planned Program as It relates to the 

■ ' ** ■ • • • ». ■ 

I ' accomplishment of* the objectives identified In Step 2 

* * • ' 

• above. , 

Step *». At the end of the monl^ taring Interva.li complete the 
Actual Program section and Indicate which objectives 
were completed and which obj^^ect Ives Were not completed. ^ 

Step 5V Recycle by seksct^lng the next object I ve(s) as I dent I-' 

L . ■ ' ' • 

* fled In Steptl above ar}d complete the Planned Progpm^ * 
section on the next monlti^rlnp sheet. * ^ 
Step 6. Continue process. ^ . 

Almost compretely /rom necessity through the Jack of alter- ' 
ves that we could afford i we were headed 1n^%he monitoring . ^ 
program development. Though there were ^ome common elements \fi 

monitoring systems, objectives, Items apd detailed records, there 

' ' . . / * ■• • ■ 

f * ■ • 

were many directions that could have been pursued. ^ 

We had long felt uncomfortable working with standardized tests 

to pyaluate our student progress. Like school administrator's ^ 

' . ' . ^ • ■ • <i ' 

everywheire, we knew there v;ere Inherent weaknesses IrV standardized 



tests . Yet;. tVi^f r 1 eyel of, a^icep tance l.s - so . hi gh tha t th^y gep- 
^eraily recfelve -^ful 1 unjuestlorued confluence of the tax i>ayer de- 
spftte theli- obvious UmltatlQris. Norm feferesnced tests may be • ' 
characterized- fai rly by the fori ov^ / • / 

-A norm referenced, test compares the performante of one 1n- 

. . • ■ ■ , ■•. • •; ^' ^ ■ - : . . • ■ •. , ■ ; • ' ; ■ • ■ 

^ dividual against the performance of iftany ot^ier Individuals, The 
poTrit at v/hlch ha 1 f of the group scores abo!(/e and the other ^a 1 f . 
scores below is the ''norb^'. Tests a that half 

> of .the students wfll score above a certain point and half bel6w. 

■■■■;■■•> ^ , ■ ■ - - • i ■ . ■ > -.. . ■ ■ ' ■ ' ^ ■ ' 

When a sItuatIon,a^rIses Iiti which more people scoV:e above than 
. belpv^^ for example^ a test mrght have to be In order 

.to achieve this standaYdlz^tldn Items chosen for the test tend to 
be Instruction probf. That Ii,/ lf an i tem Is taught sut'dessfulTy . 
to 4th graders^all over the count ry> It will not make a good test 
ItfiSm for a norm referenced t^st becaus!^ more than half ^f the 
population will score -correctly. * ^ 

Dependence ph, these teaching proof I terns may Ijave cobtrfbuted v 
to another phenomenon associated wjth norm referenced testing. One 
c^ri pr/lcllct with amazing, accuracy what a group of students will' 
.score on norm referenced tests If one knows the fol lowing; Infor- 
.matlon: ■ • ' » . 

-a. Income of father • 

a ' . . . - . r ^ '■ 

^ • b. language ability of mother - , 

c. . social-economic* position J n the community . / 



' 'For whatever reasons, In Che United Stages the children In 
w^althry commuijltles attain high scores on standardized tests and 
the children In poor communities attain low scores on' standard': 
Ized' tests. This phenon^enon has the status of legal acceptance 
In. flew York Statev/^ The standardized N^w York State tests ^dml n*- 
Jstered Jn grades 3> 6 and -9 for mathematics and reading serve as 
the 'f^ndainental Information In a booklet of Performance .Expecta- 
tions .that the State Department. published for every schooK district 
information 'concerning the district's wealth (per pupil valuation) 
aild other socio-economic data are then cbmbthed to predict what 

/•'■ ^ ^ • \ C ■ ■ ■■ ' ; ^ 

'"^ students will score on subsequent standardized tests.. In addition^ 
In 197^ the legislature passed an act called Ch^ter 2A1 of Edu- 



cation Laws of New York State. This law provides that a district 
will be reimbursed In state aid at the rat lo of 1 .25 for the per- 
centage of. students scorlrig. two more grades below level according * 
to the New York State Test. The system Is almost Infallible. 
; Wealthy districts have high scores and poor dlst relets have low 
scores. A survey of the 1 1 terature finds no notable large scale 
exceptions. All of this Is prefude to the gdnerallzatlon that 

while we are requlredto administer norm referenced tests, those 

* ■ ■ . ■ . ' * • ■ ■ • ■ " . ■* 

who administer them, general ly know that they^^wlll be of little uise . 

• ^ * '- * ^ . ■ • ' . . 

in organizing the Instructional program. On norm referenced tests, 

50% of the population must be In the sub norm position^ that I? gen- 
erally Interpreted as failure In our society. 
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- Criterion referenced tests on the other hand^, are based ori 
behavioral -objectives Stf^nents-^of expec- / 










tat f on for the performance of the learner after the spaclfied con- 

dit Ions have^been met. Attainment of the objective under those . 

* ■ » , ' • • ■ t 






specified conditions constitutes "peetlng. the crlterlop". trl- 


:! 


' ' - \ %»' ■. 


terion referenced tests measure the. perfdrmarfce of an indlylduat 
student against the performance specified In the objective. ^ 
Another way bf ex!presslng the Idea, i s to ""say that criterion 
tests measure^ a student against the gu^it^c^^^^ of comparing 
his performance against other'>fey4sV^^ ' t compares a students per- ® 
/ formance agajhst the objective. It does not compare /studerits to 
each .other or to a norm. 

Another facet of 'criterldh lesting Is that the- tests contain 
only Items that test the stated objective they are directly related 


■ • ■ " ' • «i- 

■ •. ■• ^ ^ 




td the Instructional program. / . 


' 4 \ 




In adclltlon, criterion referenced tests provide feedback con- 




* - 


cerning the Instructional progrdm. Since the objective stated 


. ■ 




student performance aft^r certain conditions were met, unsuccessful 






performance may mean ^ 






a. \further study and practice needed by student 




■ . ' A 


. b. prior learnings need to take place 




; - .- ' .. ^ 

/. ■*. ■ ' 


c. conditions stated ^in the object! ve are insufficient 

■ ' *<t , . . * . . 

'■^ .. or Inappropriate to bring about the student's behavior. 

, * j» ■ ■ . . *' ^ ■ 






Whatever the result the information given by the test is always usef^t 

■■ * " ■ ■ ' ■ . ^ ' 
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to examine student performanqe means of In.structloii or the 

• test itself. ' . " * " ■ ' 

Interpretation of 4rit^rloh referert(:ed t^^^ 
greatly from that of?*noi;m referenced tests where a student generr* - 

ally passes or falls. For criterlon'tests, his achievement Is a 

' - ■ • ■ * : , . ■ 

matching of his .per/orraance against the gpaj stated In the objective. 

Falling means that he has not .yet performed as expected; passing 

means . that he Is ready for another, objective. Criterion tests are 

locally developed to meet the needs of local students. They: may be 

adjusted accord tng Jitb heeds or according to student perforinance. 

They are flexible— - related to students^ their, goals arid their 

need3. Since criterion tests test the bbjectlviss directly, these 

tests %re very 'important In evaluating the Instructional program.^ 



Purpose 

for 
testi ng 


RequlVed Capability V 


Test Which 
Meets The • 
Requirement . 


Placement 


To test retention of knowledge and 
ski lis prerequisite to specific ob- 
jectives to be . taught In current 
courseis for 'purpose qf early rele- 
vant placement. Must pinpoint 
specific weaknesses. ^ 


x."GRT ^ NRT > , ' 
Readmlnlstratlon 
of "final test(s)«' 
from prerequisite 
course (s). 


Pretest . 


To test .dntry' knowledge and skills 
of specific, behavioral objectives 
to establish entry ^ knowledge data 
and detect students who already * 
have the required knowledge and 
skill. 

^ ^ — ^ — ^ — ' — — — — 1 — 


. xXRT NRT 
An eiqulvalent form 
of the ••final test'« 
of the tur rent 
course. 





/ ' ■ . • ■ ^ 
• . • . ■ ■ ■\ 




d 1 • ■ 

- - . 0 • . . ^ • , 
■ • V" ' ', ' w 
*. ' ' • 
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■ ■ * ■ • 
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■ * >" 
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Placement 
tOurlng 

I* f* fir iron t" 

^ Course) 

.* ^ 


f ' ,^ i ' 

■ . ' : , , . .» ^ ' 

To pinpoint* specl^fjc- weaknesse^t- - 
particularly In hlera-rchtci^l 
5 iiuations y in order to prescribe 
learning, rnater la Is /and situations 
des Ignedl- to el Im I nj^te weaknesses 
and gtrjingthen reuent Ion . ' 1 terns 
must test attainment 6f specific 
behavioral objecmyes^ 


X CRT _ NRT 

A* test .^com|3osed of ■\ 

relevant enabl Ipg 

test items. . \, 

* ,- ... 


•(, ■ ■ • 

■l 


Mastery 
Test 

s 

• 


To determine 
of a spectfic 
or a major hi 
May determine 
requisite kno\ 
or terminal o 
must test spet 
objectives. 


/ « * * 
a stud^SntS' mastery 
subject or operation 
Brarc|rilc&1 sub-unit, 
mastery of pre- 

edges and skills 
hfjectlveS*^ Items 
il'fic behavioral 


2i.^RT ^ NRT 

A ^test* composed of 

re^ 1 e;^aat term I na 1 

ol subtermlnal 
I terns . ; 


Posttest 

■ ■ ■ . 


To determine/ a students'' exit 
, knowledge In/ compar I son. I tem by 
Item, wi th cptry krtowletfge in 
order to del/ermjne the Individual 
students' Improvement- In specific 
subjects during the courise. Items 
must- test sbeclfle behavioral 
. objectives J 

/. ^ • • ^ . 


X CRT - NRT . 
Ah e<^(4lva]ent form 
of the pretest 


National 

R Anl^ \ nrr 

l\CilllS 1 11^ 


* To determi 
group of s 
from fiigh 
^11 studen 
basis . In a 
Is plotted 
curve. 


nie how a student, or 
tudents, rate on a scale, 
po 1 ow,, I n r ?J a 1 1 on to 
is on a nation-wide < 
subject area. Rating 
on .the ''bel l ;shapea" 


_ CRT X NRT 
A test composed of 
Items sampl Ing a 
subject matter de-^ 
srgned to Separate 
students by abMity 
or knowfedge. 


Ant* I t*iiHp 

Test 


To sample 
-abilities 
(on a perc 
will do we 
do poorly 
pi Ine or j 


student's aptitudes or 
In an effort to predict 
3n 1 1 1 e sea 1 e) those who 
II and those who wl 11 
In. a particular dlsclil- 
Db. 


^ CRT x^ NRT 
A test of Items : 
that has shown pre- 
dictable validity - 
and reliability. 


J 


■ \ ■ .,1 ■ 


o 
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Assigning 
Grades 
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To separate Students Into a series 
of categorfes from good (A) to 
poorf (F). The subject matt%r Is ' 
sampled. ^Emphasis is quite* often 
on testing In an effort to separate 
students ^rom "F*' students. 



J CRT X NRT 
A test composed 
of Items sampl Ing 
prevlcjjpsly taught 
subject matter. 



. In 1971 Dr. Brleg^r and I met with Or. Robert O'ReMly, Chief 

of the Bureau of Research' and Cultural Affairs, New York State Ed-' 

• *• . . ' ■ ■ ♦ ' ^ 

Mcatl'on Department and Dr. William Gorth, School of Bduoatlon, r 

. • •■ . ■ • . " ■ . • 

UhWersI ty of Hassachuse.tts. At tttls meeting we learned of a com- 

pdterlze^ monitoring system V/hi^h they ha ^ developed With Dr. Plnsky / 

In Cal ifofnla. ^ The cpmputier offered the Ingredients needed for our , ' 

Situation. (This will be dlscussedMn detail In the section on 

Process of Implementation) * That Is, large-*sicale capact ty for perA 

foriT](Tng', scorlng^^^^^^^ clerical functions with great speed^for dV^r-? 

1^ flight ;turnrarounxJ of results -for teachers* ^ .^ 

The requirements qf, the Computer, hoWever, were that Inputs, 

■ - . ^ ■ ^ ° - ' • • - . 

I.e. , objecjtlves, ltems|^tests, answers, etc. had to be precisely 
and exact|.y detailed 'In a preset form and sequence* It became this 
writer^ task .to organize a staff of principals and teachers to 

a. • v/rlte and select objectives 
^ b. develop test Items for those objectives 

c. d,evelop tests . ^ 



InstrAJCtlonai Module Criterion Referenced Testing , New York 
State Ecjucatioji Department , Bureau of Research, Albany, 1^71. 
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develop manuals for t;eacfie^rs^ parents ^nd students; - \ 

e, develop cross reference guides which Indlcatejd wfiere Jri 

• ■ ■ » . . >. ■ • ■ . . 

^ the rnaterlals avatjabre In our schools, a particular ob- 
jective w^s taught. 

f , code objectives for the computer ^ , • 

g, arrange for programming 



h. prepare teachers - . 

;> « ■ ^ ' ^ . "I . • ■ • . 
I. distribute all required materials • ^ 

J. coordinate test^lng, , col lectloti/ scoring and return 

_ k. monitoring of tKe monitoring program to evaluate Its 

contrlbytlor/ to the educational* prograjp 

It was determined that the first trial of the program v;ould be 

In grades 4, 5 and 6 In mathematics. Mathematics was chosed be- 

cause ' 

a. we were In great need to Improve our teaching of mathe- 
matics. . ' ' 

b. more work had been done with behavioral objecjtlyes and 
test Items In mathematics than for other subjects. 

c. the precision of- mathematics seemed to off er greater •hope 
, ' y for success In the f irst trial. ' ^- ^ 

Having elected to employ a criterion-referenced testing system 
^to monitor student progress m the Brerftwooc^ School Sr we were given, 
the opportunity to re-examine our Instructional program In terms of 
needs of the students and communltyjV VJ!e did not pursue this In a 



fashidp that would hay^ satisfied Dt. Stuf,f'lebeam, yet «fe did eX7 
amine quite carefgilly where we stood In relation to what It was we 
hoped to achlevcK* Some .of the factors we considered were: 

y. Community Expectations . Brentwooxl parents, It. was soon 
. observed, believed In educatloifi as a way of Improving 
. 'fltianblal and social standing for their children. They 
. had hot yet ^read Professor Jencks* ^ % 
2.- Student Charact erlsfalcs I Academically 'trentwodd. students 
/ followed a statewlc|e*pattern. 3rd grade performance was 
hlg^ but declined at a faster than statewide rate at 6th 

grade >dnd 9th >grade level v An Immedlaife gqar became t.he 

* 111'. ' * ■"' ■ . 

" lm||rbyemeM of perfbrmance *on 6th grade tests to at .least 

^tafteWlde levels. . ' • / V . 

.■ ■' ■■ . • ■ ■ ^ ■ \ 

■ ' '■*•■"■■■, ^ ■ 

• 3. Sttideft t Interests . There Is a strong.f a I thJn education 

on f|:he%art of primary grade students! They believe they^ 

■ * ■ •• " I ■ '^'l-' ■ . ' " ' ' ■ ■ ■ - ■ ■ ■ ■ 

« carlh^rr^ and that If they learn they wl M succeed. This 

faith 4lr|inlshes witfi age but remains strong* 
Perfd.riyraripe Characteristics of Graduates. 
' a^ 25%igo to four' year col leges 
:b.-v25%igb on to two year colleges and vocational tralnjng 
* V c. 50% join the work force, military or marry to/ralse a 
fami ly "Immediately / 



^ Crlstopher Jencks , et al.. Inequality A Reassessment of the 
\ Effect of Family and Schooling in America, Basic Books, Inc, 
Pub., New York, 1972. 



In reviewing community andcStudent needs It became clear tliat 
there was a strong desire and need to perforin well academically. 
The d;l lemma occurred because th^^ communl ty standard for judging 
suo6;^5S Was a norm-refer^^ed test; ~ The 'Uew York State Tdst 
administered at grades 3f 6, and 9. The results of this test are 
annually published In the local heWspap^. Comniunrtlds ja're ^TlJ'' 
listed ar>d eas I ly matched against each other. / " 

Jn addition to Irnproved! student 1 earning. denionstra ted on 
crl ter^oil j^ests » we would have to kt^^levi^ Improved scores on stand 
ardlzdd tests at the 6th grade level^. We made this a fhree year 
goal of'the program. ' ^ 



Xt. The Program . 

. " • . " ■ ■ • • . •■ ' * 

At an earl'Y meeting I' heard Or** WMHam Gorth discuss the 
problem of developing useful Information about student performance 
as one of the central Issues In currlc<ilum developrtent and adjust- 
ment. He posed the question In terms of graphs; * 

Student Score . / 

100/ 



50 




Sept.. V Jan.^ Jun^ 

In this graph we see lltustrated the sltuatltm*-feh6t fto of 
us hope for when we teach. ^Many teachers believe It Is what 
actually happefis. Thls> of course, Is the Ideal learning; pattern 

In which a student comes to class not knoWlrig v;hat is In the course. 

■ -•>-■' 

At the starting point In September, the teacher teaches this par- 
ticular Mjndf^,. The student proceeds to go from near zero not know- 
Ing to nelr^^lQO knowing. Furthermore, he does not forget. He keeps 
on knowing overtime. This probably doesn't occur regularly In In- 
struction but most teachers are not In a position of knowing. They 
do not have a pretest to measure what- the student kpew on entering 
the course. They do not have an objectives based test to measure 
with precision an objective of the course and they do not have * 





... ' . 


' *■ . • ' » ' ' ' ' • ■ . ^ ■ 


i. 

t ' 5 
i 


a 








■ ••' . • ^' 


• ■ y 






'■ ;■' ■ : . " 


folltsw-up tests on that same objective for measuring retention ^ . 
overtime. ' ' a " -^-^ 






Another pi)ss lb ill ty- 






^. ^^udent Score ^ > • , • 






' 100 








' •. . 50. 

0 




• ■ i 






Sept. Jan. June - ^ 




/ 


. ) In this " 
«(acb time the 

f ' ' 


rittl^ bit at a time" learnlngV the stHdent learns 
teacher ^teaches^^ ^ He doejs; not forget and the In- 






f erements are pven. 'Again, this Is probably not a regular pattern. 






Even so It wpuld\ require a pretest, test and retention test pat- 






tern to know that this Is occurring. 




>* ■ 


^ Student Score » " , 




> 


• 100 

\; 

; . 50 
' ' 0 








• 


Sept. Jan. June 






In this pattern the stud^t came Into the course knowing this 




* . 
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part. J He did ndt forget It and regular, Instruction Had no effect. 
With pretest Information and objective' follow-up tests the teacher 
would be In a position to-wo on to'^iteachlng something else to a 
student who exhllilted such^^ctifiracterlstlcs.;' . ' 
Stadent^ Score • * 



100 



50 



I r 

# t 

# I 

/■V 



t 
i 

0 



I 
I 

i 
t 



Septr Jan. \ June 

In this -pattern a student enters not knowing the .object fve* He 
Is taught, and learns It very well. However, at jthg next testing he 
scores near. zero again. This pattern Indicates forgetting. This 
system of testing Indicates that re-teachIng Is no\5 necessary. 
Student Score- 

ioo4 — — --.--V 



50 



Sept. ■ ,Jan. June 

In thU pattern, one that every teacher has prolsably felt, the 



24 



student enters the class knowing the objective. The teacher teac^^^ 
•for this objective and the student promptly becomes confused. This 
situation Is probably IfjeVl table In ^very class. It really poses 
no problem unless the teacher does not gain the Information that 
the student Is now confused. This requires the pretest test*' and re- 
t^st/pattern/ /* ' v 

Npne|0f ^hls Is new. Experts In- evaluation have long preached 
It as a, way of knowing the effects of our Instruction. The problem 
has always been that ^ 

a. stating the objective 

b. making the tests 
»c. scoring tests 

d. writing reports for the student and qther 
Interested In hjs progress 
have been top time consuming dnd too complex to be done regularly 
by an Individual teachejf. < . . 

. Comprehensive Achievement Mont tortng, which was outlined for 

■ t ."> ■ ,, ■ ■.■ . • ' ■ ' " ■ 

^us by Dr. O'Rel 1 ly. and #r. Gorth, was a system fqr employing the 

' ■ ^ ■ * ' - j ■ ■ ' \ ' . " 

computer to keep track of objectives and student perfarmah to 

write reports, to keep cumulative Information from test to , test 

for Individual objectives or for total test. ' 

-■■■■■*■ • 
■'In the following pages I will attempt t<J , describe and explain 

\ ^ . ■ ■ ' ' ■ ' ' ' ' ■ ' 

^the pomprehenslve Achievement Monitoring System that wa? developed 
In 6rentwooci. )n 'x>ur- schoal district we (In the happy American 




tradition of cr^a'tlnjj.^gferonyms for every th I ngV'ca lied It B^E.^j^T: 
Brentwood Educatldhal System for testing. . r ' . ' 

^ The components of the:-sy St em are:' > , , 

U Behavioral Qljjectlves ' , ' 

/ ■ ■ . ■ ■ • ^ 'I * . ■■ ■ . ^ « 

Test Item bank ^' ^ . * ' * 

3* Test construction program • / ,\ ^ . ; , ^. 

. Six test forms^ for^ each leyel for , each* semester . . ; ^- ' 

5. Answer keys * ' 

6« .Cross reference guides ^ \, . 

7. -Manuals for students, parents, visitors ^ ■ ,5 
8* ' Forms for entering Information Into the various , . 
daft'aN^anks: r V . / 

^ a. Vo&Jectlve entry forrtr / * . , V 

• b, .Item 0n try form ^ v . \ 

' c. Student update - add and drbp form ^ 
d. test construdtlori' form .v ^ ^ 

Administrators and teachei^|j our district (sect li)n^ ^o 
plementatlon wMI e;^pla in process) selected jand v/rote bdhavldfc^^l 

objectives ^ipp'roRi'Iate fdr bur students fair grades-H> Si ^'nd'o. ' 

. ■■■■ ■ - ■ ' ". ■ ' ■■ ^ ; '■■ , Tv ■ :/? V> , ■ ' . ' ■ ' '■ ' ■ ■ 
Eight basic strancfs vJere developed for the program: , V V 



K Numbers i^Numeration ^ 
2, Basic Operations v/itH N^^ural Numbers 
3» Rational NumheVs ^ 

Decimals ^ ' ; . ^ ' 



. ♦ 11 

^ ^ / II 

' * II 


■ ' ' •' V ' -' .^ ' ■* ■■ . ■ - ■ ' , . ■ ' ' ••*'.. ■ . % ■ j 


. ■■ ■ V ■ 




" " ' ^' ' ■•• ^ ■ ■■ ' ■ . . ' ': ' ; \ - V. '. .1 




^ Ir 

• II 

* ■ II 

• ' ** 11 
II 

' " * II 


, 6, \tord Problefns ^ * * . J 




* " V II 


Measurement : 1 




f II 


'.• -8. JGepmetry .' V ■ ■ . v • ' " . ^ ' \* ■•'-^I 




II 

■* •11 


For S^df these topics discrete objectives'' were written, (see , 1 


♦ ■• 


■ ^ ■ . 11 
II 


Jn order to keep track of the objectives and to have the com^ 


Q ■ 




• • ■ " . ' - ■ • 1 

)puter manipulate them a coding system had to be developed • A 

». • • . , . ''&■"■.,•>.■ ■ ^ • 1 




; ■ * 1 


simple procedure vfas developed. . ^ 




- * ■ • t 


1 or 00 ' »00 - W V ' 
Math Grade Topic Specific Objective ^ 




"1 

*' 1 


so that objectlve-#01- B^-02 0^ / - ^ 
a, math objective . 

b* fourth gr^de "" • ^ > v 
[ ""*"^cT second topic (basic operations jwlth natural numbers) 
j first .objective ^ ^ ^ 






j For this mpnitoring system a bank with a ;||JnImum of four ^^^^^ 






r items Is required for each objective* the quail ty of theSe test ' 






Items can be determined at the end of a. semester^or year through a ^ 






1 detailed Item analysis supplied by the computer. However, In the^ 






1 . ' . • ' • . . ■ 

1 first wr Itl ng of test I tems the f ol lowing procedure helps to assure * 






„ that items will reasonably test the object rves they purport to test. 






1 _ a. ' a good objective has ImpMcite a test Item irtcVuded. 


1 


' .... " ■ . 


Example: OBJECTIVE -'The student will demonstrate the 


1 

I ♦ 
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,04-02-006-1^ Mufl-jpiieattbn 



X OBJECTIVE: Students w 1 1 jr se I ect the natuf a 1 'number which Is 

; the correct answer tp> :glven' mult ipltcatlon pr^^ 

' V \ • The multiplication probtem. maybe of two types: 

IV the multl'pl tcand Is a 3-dIgIt number and the ' 
:• .multlprier Is a 2-dig'lt number^, * 

• • 2) this multlpl Ter and multlpl icand are both 2Td 1311" 

•■- numbers. ■. ' ' 



EXAMPLE: 



253 . 

?< 32 ■ , ■ 

(A) 9096 ^' 
(C) 8096 



(B) 7090 

' ' '■, 

(D) 8090 



04-02-00:7-00 D I v is Ion 



Second' semester 



' OBJECTIVE: Students wl I T select the correct anlwer, to a 
: . , given division problem of natural number^. The 

divisor may have one or two digits, and -the ' 
. dividend wl have, four o.r fewer digits. ' 



EXAMPLE: 



15)3926 
(A) 265 Rl 2 
(C) 65'RI I 



(B> 261 R5" 
.•■ (p> 261. RII 

Second ^g^mester 



.4 .■>. 



RATIONAL NUMBERS ' 



• 04-03-00 1-00 Egulvalent fract I ons 



,>C" OBJECTIVE:" Students V/1 1 1 select the proper fraction. that Is 

not equivalent. The denominators of al I 
equivalent fractions will be less than or equal 

■ . to iqo. - ; - - ■ : ■ . . ^ 

'■; EXAMPLE: Which fi-actlon Is not equivalent to I , 

V. . ' ■ . ' . ■ . . . "2^ 



(A) 4 
8 

(C) 8 



(B) 5 

(D) ro 
So"* 
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(fig. 1) - Objective Bank 



abfUty tb count to 100. 
TEST ITEM - 5. 6, 

' 5 / '. ' ■ ■•. 

(b) 7 . ■ 

' (c) 8 ■ ■ ' ' \- 

(d) 9 

If the test Item closely reflects the objective. It Is 
likely a suitable item* . ' ' 
Reading an^Tysi 5 IS sometimes needed to determine dfp- / 
proprTal^ness of " 

1. vbcabaiaFy in-^the It^ ; 

2. sentence length in the item 

3. sentence complexity in the item/ 

In the multiple choice format the qual i ty of ^the distract 
OPS }9 an important part of the qual ity of the test item. 
The two most common problems with distr'actors sre 

1. , The distractor Is so .close to being the correct 
answer that it may not discriminate between a 

/ simple student error: and a lack of understanding 

■ ,j ■ ■ -■ . ■ " . ' " • ■ ' 

^2* The distractor is' so obvious that it/could not 

- • . . - ^ . • ' " '/ • 

possibly be considered as a suitable answer by 

any student. 

The multipl^e choice format was chosen for the testing 
'progr'am because it irtost easily lends itself to computer- 



tzed testing and scoring. In a well constructed' Item In 
':^.fnathenatjcs, the chances of a student's- guessing the ^ 
correct answer ls'25%. However, keeplri^ In mind that tfie 
student Will be tested s fx times In each serriester, we can 
figure the guessing odds another way. The chances that a ' 

student wHI guess the bori^ct answer on two tests In a 

.■■<• ■ . ' ■ ■ ■ - - * . • — ■ , 

row are ,0625 which Is. a very small Ichance Indeed, (see? 

: ' ■' -f^tg. 2) ■ ■■■ - r ' ■ ■ ■ ■ 

NaturVHy It was necessary to develop a coding system to enter 

te|st Items In the computer ban P;, Test Items were coded as follows: 

?[ 00 . 00 * jOO 00 00 V 

• V Math^ :;(|rade' Topic ObjiectiVe Item'# 

' -t •* ■ ■ . * • 

In-order to make a monitbrlng system from a b^nk of objectives ar\d 
a bank of test Items a test construction program Is necessary. The 
sy-st em decided upon for Brentwood has been to first develop' the 
trend testing system to be followed and supported two years later 

- by a mastery festlng system. . • 

The monitoring system Initial ly designed to take advantage of • 

•^the eomputer'§ speed and capacity for storing and reporting on large 
numbers of variables. In addition^ the CAM System makes it possible 
td'test students frequent ly/on^a^rge number of objectives whilev 
using a limited number of test Items. There are a number of reasons 

that this Is desirable. Oniexample Is -the length of the test, f The 

. o • . } L ^ . ■ ■ * . ■ ' ■ ' ■ ' ■ 

mohltoring system, which asieL six tests per semester, wl 1 1 have six 
different test forms. If four test Items are available for eaoi. 



04r02-Q06-00 . Multiplication ■ 



\\\ 



OBJECTIVE: Stud«ints wi ll select the natural number which Is * 
. the (forrect answer to3 given multlplIcatJon problem 

. The riultlpllcatlon probiern.^m^ 
• 'I) ^'he multlpHcandl Is a 3-dIglt number and the 
• * iiultlpMler Is a Zrdlgit number; 

' "■ ' 2) the multiplier and ,multlp I Icahd are both 2-dIglt 
\ ■ lumbers. ' ■ ■ * ■ 

■■■•V- ■■■ ■ ■ ■ ■ ■ ^ ■ 



^ EXAf-IPLE: 



6 253 
X 32 



04-02-007-00 Division 



(A) 
(C) 



096 
18096 



(B) 7p90 
(0) 8090 

Second semester 



EXAMPLE: 



OBJECTIVE: , Stutlents wi l l select the correct' answer to a 

g I V jn d I V I s I on p rob I em of natura 1 numbers . The 
divisor may have one or two d||L|ts, and the 
dividend "will have four o.r few§^ digits. 



15J3926 
(Af 265 Rl 2 
.(Q) '65 Rl l 



(B) 261 R5 
f Jd) 261 RM 



RATlONAIr NUMBERS 



Second" semester 



04-03-001 -0 0 Equlvatent fract ijbns 

.OBJECTIVE;, 



: ^ 
- I" 



Students Will select the proper fraction that ^j^^ 
not equivalent. The dencrnlnators of all 
equivalent fractlons^ will be less than or equa'l ' 
to 100. • 



yC EJ^AMPLE: Which fraction Is not equivalent to T , 



(A) 4 
■ 8 



(B) 5 

^ 9 



(C) 8 

T6 i^' 



(D) ira 

^ 20 
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(f Igr. 2) - item Ban1< 



objective, each test will test h/6 of the objectives. Stated 
another way, over a course of six tests an Individual student will 
be tested four times on each objective* The whole class will be 
tested on all of tRe objectives/ ? After the, second test all students 

* . _ ■' . ■■ ■ 

In the class will have been tested at least one time on each ob'^ 
jective^ selected, ^ • * * 

The computer 'program for assigning students to tests works 
this way* / < i 

Iv There win be six tests* | 
2. There wil l be six forms for each test to, be sure that 
I" ^ students do not*tgke a form of the test ihore .than on^ 

time and to assure that each' stM^ent'Vlll l^ak^ all the . 
' ' test forms a table 'I? deyel'oped^ first six groups are , 
•created In) each clas§room.£, Then they ar^asslgned test 
forms by computer In /he following manner: 



Group % 


TestsS 


' 2 


3 


h 


•5 


6 


T 


droup 2- - 


' if . '^ 


^ \ 




■5 


6 


1 


1 


Group 3 


Tests 


5 


6 


1 


'2 


3 


Group k 


Tests 


5 


6 




2 


3 


i» 


Group 5 


Tests. 


6 


i 




, 3 


if 


■ 5' i 


Group 6 


Tests' 


1 


.1 






5 


6' 



There are many advantages to this kinol^f sampling. First, 
• It practically eliminates the problem of cheating on tests. Since 
students cannot Be' sure that a n^elghbor has the same test form, 
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there Is Mttle^olnt in cheating. Secondly,the tests can be of 
reasonable duration-. If aU sUyJerrt^ were tested on all objec'tlvesT 
In ea<;hc t<^st the test would^lp this case^, necessarily jbe 1/3 longer. 
The testsi which are given twelve times a year, last from 30 to 

■ -. ■ ^ ' ' ^ ^ ■ • ■ 

ml^nutes. Economy in , time required is important when thisre Is such 
a test, frequency. 

, The computer will assign student groups to each test, form; it 
will also/ assign objectives and test' items 6n each of the six test 
fotms^ (See Appendix G, N.Y.S.E.D. Cc^bol format) 

the -resulting six test forms in multiple choice format, (See 
Appendix H) are then available for student use: The following kinds 
of question j(^e Tig. 3) • There would be 25 on' each test that would 
. be responded to. This is one of .the two kinds of testing systems ^ 
developed within the BEST Program^ ' The definition of objectives 
by grade or difflcul)ty level, the computerized random assignmerjt 
^ "^of test forms for the purpose of sampl I ng student performance regu- 
larly on the total set of objectives; this is the Trend test aspect 
of the program. ^ A complete' set of the Trend tests In the BEST 
system may be seen in appendix H. 

Mastery tests are different in that one objective' Is tested , 
five or more times on one test to establish a level of mastery* 
such as five out of five, four out of five, etc. The Brentwood 
set of mastery tests may be seen In appendix D. 
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A> 3 LB. lil OZ. CB> i» LB. 1 OZ. CC) - 4 LBS. 10 OZV CD) ^1 LB. OZ. 



ONE SCHOOL BUS. CAN CARftY il7 STUDENTS .. HOW MANY STUDENTiS CAN 
7 SCHOOL BUSES CARRY? 



CA)*^. 7 



CBO 289 



<c-) 329 



CD) m 



IF JOHN BOUGHT 213 JELJ-Y BEANS EACH DAY, HOW MANY JELLY BEANS WOULD He' , 
HAVE IN 67 DAYS? _ ' 



CA) 4^271 



CB) 14,161 



CQ> . lit, 271 CD) i»,l6l 



VJHiCH FRACTION IS NOT EQUIVALENT TO | ? 



CA) -6 

. i5 



f6 



CC) 12 
32 
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CD) 6 
15 



• (ff9..3) 
Math Monitoring Test 
sample page 



A c1as£^ of 3Q students would have groups of five students 
taking ea^h of the six test formsy The student will h^ve a pre^- 
printed answer s^eet. (see frg, k) . Across'' the top Is listed 
the student^ riumljer (assigned by census department when student 
enrolls), student (name, school number, grade and homeroom number. 
The teacher Is #1 so designated by number, usual ly the s^me as t^he 
homeroom number. 

Spaces a re ^provided for the student to mark the answers he 
believes to be cbrrect. A regular number pencil Is used. Our 

first attempts required use of a grease pencil. Somehow, we could 

7.- .V.I . • . . . ' 

never have enough of these on hand when they were needed. This 
p rob 1 em d I sappea red when the .prqg nam ms x;h5mg ed .to use or d I na ry 
pencils. . * 

The bottom,. ha If of the page l ists the objective numbers for 
the course th^t this student Is currentlV enrolled in. The teache 
marks or has the students mark the numbers of the objectives that 
have beet} taught since the last trend test. 

The entire proce'dure of coding for the computer was handled 
In the curriculum office; The well defined procedure has evolved 
the foljowing, presently used forms, which organize Information 
for the key punch operators at the dat^ processing office: • 
^ 1. Objective Bank Updating Form (see fig. 5) 

The action 'box offers three posslblitles - 0 « delete, 
I = new, meaning to add this object I ve, and 1 = change. 
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[;T 1 STUDENT ANSWER 'J&SCTION. INSTRUCTIONS FOR FILUNG Cut THIS, SECTION 

1. CHCOStt-C.VLY ONE GCX FOR EACH QUESTION. - - 

2, USE A.NO. 2 PENCIL TO MARK THE BOX OF YOUR CHOICE. 
' 3.- SHOULD YOU WISH TO CHANGE YOUR CHOICE, COWPl^ETELY ERASE THE OLD..MARK. 
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WT OBJECTIVe SECTION. INSTRUCTIONS FOR FALLING OUT THIS SECTION. 



1. LISTED BELOW ARE THE OBJECTI^J'ES FOR THE CURRENT SEMESTER 

2. USING A NO. 2 PENCIL. MARK THE BO)C LABELED YES FOR EACH 
OBJE'CTIVE taught <^IMrF THg PRFVIOUS inEm TEST. 

3. SHOULD YOU WISH Tt) CHANGE A MARK, COMPLETELY ERASE THE OLD MARK. 
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CSJECTIVe 
NJMBER 
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TAUGHT 
YES NO 
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OBJECTIVE 
NUMBER 
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OBJECTIVE 
NUMBER 

I4201C07 
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14203003 
14203007 
14206001 
14206008 
14207004 
14208003 



(fig; A) 0 ■'!; 

Student Answer Sljpetp^ 
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meaning pnly the changei indicated fn the text Is to be 


■ i ' 




made. The objective number was previously discussed. In 






'the objective number the a pea refers to math, readlnaor 
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*, • 


other subject; In Brentwoody math Is always 1. 'Level 


f '■ 




means grade level or difficulty level. Topic refers to 


I 




one of the eight strands of the program. . Number Identi- 






fies the position of the object [n the sequence of objects 






^ for this topic. Four boxes for sub-top'^Ics ttave hot been 
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used In the elementary program up to thl:s point. The ; > 
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source box provides for a code . to I dent I fv the source of ' 






the object. 1 stands for Brentwood, 2' stands for New York 






Starte ^Education Department, etc. The next tWo boxes^deh- 


V- ' J 


■. , 


••^ tify the number of -lines of text In this obfectlve. The 


■ ■ ■ ^ 




text of the objective Is then Writtenf In, one box for 




' , * "' 


each letter, space, punctuation, etc. Thlii^Q.rm has Been 






widely used in our reading program. It does not provide 


.j- 




for mathematical symbols and and geometric figures. Untjl 






' we can code those figures for the computer, they wl 1 1 ' ' 
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have to be drawn by hand and stored on paper. 






1. (tem Bank Updating Form (see fig. 6) 






This form fol lows essential ly the same procedure for ^ 
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adding items, deleting items and changing items. 
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Storing objectives and Items Is half of the storage bank 
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problem* The 'other half Is recording anS keeping current 
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; ; /'o n (f ig/ 6) - F<5rm used for ea^^^^ 

- . • V ■ . OvJ or delTettng test I-tems 



the minions of bits of Information about students* 
Update Student Bank Form (see fig 7) 

.This form provides the means to enter^, delete anB change • . 
data about students in the progracn$. The first llne\bf 
Information identifies the student "by name, number, school " 
and grade. It afso proyide§ space to Indfis^tejwh ether this 
Is an: addition, deletiqn or change. Four Identical blocks 
are civaTlable below because students In BreYitwood may po- 
tentially be entered in four areas; for example: 1 - Math, 
2 - Reading, 3 .- Spe^liing and ^ - Language Ar'ts^ Each of 
these areas wpu I'd heed information to indicate 

a, area' (which subject) , , ' 

b, teacher • /i 
"c. course ' 

d, section » , " 

e, group 
st^c 

frofli one abl llty" level to another. The teacher may change 
the level at which the student is being tested by simply in 
diCatlnp the new studeht. level for the test period coming. 
Teacher Names^ Form . (see fig. 8) , ^ . y , ■ . 

This form iS'for coding ^teacKer name^ and numbers for, each 

^ "* ' - . • 

school . ' 



Each test ^period, the studfent has an opportunity to move 
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(fig^ 7) - Form ijsed for entering, - 
changing or alterliig Jn-\ 
V formatron about ^a student 

. ^ In the- program. 
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42 , (fig. 8) Form for generating computer 

: Ustlng of teachers • In program, 
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5. Test Construction is^e f ig. ^) ^; - ' * 
This indicates another immediate prQduct from operatfori of 






. ■ ■ ■ ' ' " ■'. ■ ■ ' ••V- • " ' ■ ^' . ' 
- the computer program. Student Inforrriatlon produces fjrst 




■ ' •• » ■ . • •. 


• ' the answer sheet- ObjectlvW, i tern and test form informa- • 
tlon produces a testing pf Itmes In sequence as they will - 
appear; on each test form. Once this form has been produced, 
, - it is a srmple tdsk ijq retrieve the item cards and set theim 

^ ^ up s in test form. These are tjhen photographed and the tests 

' ■ ' ■ ', .- . ■■ - • ' ■ * ■ ■ ■ ■ , ■ ■ .1 

are produced. ■ . ^ . 1 
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Indicating test ItQjnis usedj 
and ^ pofs ! 1 1 on occupied - oh 
each tfl^ 




niv The Reports > A test Schedu lets pub 1 1 shelds; pr i o r to the 
beginning of the year. On the day of the tes$r the team leader and 

an aide and teacher distribute answer^^et^ being very careful 

' ■ ' • ' ** 

'■■ ■'• . ■ ■ ■ . • 

each student has the precise test called for by the students com^ 

puter pre-prlnted answer sheet. Students take The test. The tests 

are col lec ted and sent to the district curricul urn office where they 

ar^ spot checked. The entire district's tests are then sent^t^ the 

data p,rocessing center >^here they are read by the op-sCan machine 

which produces key punched cards which" are then procjessed by the 

computer. By nine o*<:lock ^he fol lowing morning^ the r'feports are 

picked up and dlstrijpUted to the schools by the curriculum office. 

The follaving. repot ts are the computer in .the 

Brentwood Educational System for Testing: 

1. Trerfd Test StXident RepoW ^see"^ fig. 10) 

This reporjt is designed to he taken home by tshe student. 

The Information is rather straightforward* All the nec- 

/ essary Information concern ihg the student name, number 

' , ■ . - ■ ■■■■ ' ' ■ - 

and sectibn:^ his teacher Vs number and school number at 
the left from- top to bottom are l isted, and The topics 

^ • ' ■ ^ ^ ■ •■- ■..•■^ -H:-- -:' •., . ■ \ 

and oBjective numb&rs that were testted. The students had** 
previously taken home the Manual for Students and Par^ents 
(seie appendix B). Using the manual , the students and . 
parents can find the specific objective that was tested. v 
.\The manual always gives a test item sample v/ith each ob- 
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(fig, 10) Moriftoring system report 
• for the students / ' 
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jectlve statement. In matching* the student repoft with the , 
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manual the strident, teach^f^and parent covexpbj Active by 
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I' ' . ' . . 


• objective by objective the student |s performance on the 






test of the previous day. The code at the bottom^ of thfe 
page Indicates that a ^ / ^ ^ 
C means correct 




I ' - * ■ •- ■ . 

*' - 

}■:■'■. ' 
\ - . -■ ' . ^ 


W means wrong . . 




N means the student did not answer the question ^ 

^ T means that this obj^ectlve has be^j^ taught 

i ■ . » . ■ • .. * " 

> Implications for decision making will be discussed late 
^ The bottom of the student report also gives a percent score 
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j on all the 'litems -and ^awther percerrt -score for those^test 
. Items x>n >/hich the teacher^ have indicated that teaching ? 

has taken place. 
2. Content Summary Report (see fig, 11) ' 
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This report Is designed to give thie teacher information 

. •* . ■ . 

about group performance on the objectives that were tested 




I Q * ■ 

1 *. ; . • 

^ - -. ■ ^ 


* down the left column the objectives are listed. Again, 

the teacher has a manual which is a key for providing ' 

■ • ■ , J. 
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1 statements of the objectives and sample test items.' The 

• ■■ ' . ■■■ v . i . . : 
figure under column A tells the number of sjigsponses that 
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were made for tha\ objective, Cdlumn B give§ the percent 
correct of those responses. Column C lists the number of 
responses on objectives that the teacher had indicated as 
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• taught lij'd Column D Indicates the percent correct dn the 
objectives that had been taught*' The Information Is, 
cumulative for each test period for the entire semester. 

IV Teacher Summary Report (see f Ig^ 12) * . ^ . 
V^Thiis report lists all the students In a teacher's class 
with a score on the total test. 

Cont ent Summary Report for School Principal (^ee fig.llj) 
This report Is organized in the same pattern as the 
teacher's content summary^ report. However, the Rrlncipal's 
' report tells how all the students at a given level or Igrade 
f have performed on each objecfive ,for the level or gradis. 

Besides giving the principaV a feeling of ^achievement c^r \ 

lack of achievement at each level, this report prc?Vides 

, ■ " - . ' >• ■■/. ■ •■. - '^ 

■* him with some specific Infprmatiqn such as: 

a. .How many students, have been taught for a particular 

objective at each level. / . . 

b. How well groups of students ^re performing on those 

' objectives which have been taught, 
f - In addition, , this kind' of •repoi'ti^an be used by teachers 
' j^t a gra'de level /for re-grouping) students for review or 
* introduction of ma'terials accord! ing to similar deficiencies 

^ ^ ■ • ' ' ■ . , ' ■ ■ I : ■ ', - ■ 

or accomplishments.' Altogether the information on this 

' ■■ ^' ,' '. ■ ■ ■ - 

report helps provide a basis for discussion about student . 
growth in an academic subject. ■ . 
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•5* Content SuAiimary Report for AdmlnlstratTon , (se? ffg, 

This report Is simflar to the principals report. However, 
It provides information dlstrlctwlde about student per- 
formance at Arade 1 eve Is on different levels. This In- 
formation Is |nvaluable In considering questl^^^ 
a» effect Ivertess of specific materials 
b. effeetlVjen^ss of programs, .methods 

c# 4Ppropriatdness o*f particular ,objectI>>es or tiest Items. 
V d. sequence of|mater ia Is and problems . 

Curriculum pTanrters^ too often must make judgements about 

programs withoutl information systems* Without such a 

\ " • ■ 

system there is ciDnsldefably less rational basis. ' 
. Student Performance' Analysis by Class / (see fig; 15) 
This report which 1^1 s provided to each teacher after eadh 
te*st, twelve times a year is very Important to most of our 
teachersJ It' lists all the students in a teacher's class. ' 
Across t^hl top of the page each objettiS^e for the. level Is 
prfntfed. SFq the right of ^ each student •$ name there appears 
the desibn|tion " " ^ 

C « Corrlct for that objective ' ^ 
^ W w Wrc|ig|for that objective . 
N « Mo response for that objective 
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Monitoring System report for Central Administration 
1 Isf Ing student performance ohieach object fVe for 
all students In school district at each level 
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-Mbnltorlng System Repeat listing Individual 
sttidents.and performan,5e^on each objective 
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■ ■ ■' ;• ■ ■ . \- ■■ 


. ; by objective and the potential for grpuprng according to' , 
slmi lari ties and differences^^in perfortnance for tbe yanfous ^^. 
pbjectiA^es., Debdrahi^ Richard, Ray and Karolyn vyould .seem , , 
to be a naturail study group for the review of objective . ? 


' V ■ 


* * s ; * 15101003. Each of these students scored a wrong answeir - ^^^^^^^ ^^^^^ ^^^ i 




V . though thfs objective had been taugl^t. Many other , such ' -f ' . j 

"natural" groups and combinations are read! ly apparerft,* - ■ 

* / * - ■ ' ■ &. ■. • 

V Th'Is report is a good example of teacher cphtrifauttonv 




•to the improvement of /a program. All this Information v/as ^ ^ 




'available before th]s report wa% devi^Toped, However, 




teachers repeatedly requested this format because of the - 1 


■ - » 


visual advantage it provided. They Insisted that looking j 
V at class perfornrance "at a glance^' would make all the in- '' 5 
. ^format Ion more usable. When the data processing ^center ^ 
> ; finally succeeded in produci^j^the report there was^a , , 




quick acceptance of the report and use of the information. • ' i 

' ' ■ * • . ■ / , ^ ■ 1 
V Also after each test, each school is provided with a ■ \ 






7. Student Bank Update, (see fig.. 16) 1^ - 

• — ^ — ' . ^ — — ' ' iv * ^ - . .- ; 




■ ^ This is a Itsting of stud^t^ in the program after the / ] 


i * 

: ■ ' : ' . ■ ^ 


changes have been made for this period; Team leaders 

- ■ ■ ^ • ■ • ; .■ \ - : . ■ . . ■ •. . • i 

(explained ii> section on impMenentatioh) review the list 
tcr make sure al 1 students ai^e'^correct ly ^ntered. ^. 


■ ."' ■ • ' 


- Sirnllar print-outs are available on request .from the ■ \ 


■. ■ ' f . 

^' ' 


' bank of objectives and test itemsl . 
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Student .j^ank 



!5r,i • I 



Itfen)- Ana lysfS i> (see f tg / f?) ^ , 
At the end of eacti semester; i the comptiter program auto- 
jnfiatical ly syppHes to the' central off iqe an' Item Analysis. 
TMs repont lists a the test item riu left 
cotumn, Across'^the top are 1 isted the choices that were 
ayaTlable to the students. Ho response meaning the 
Student chose npt to answer quesftlon. A, B,C>D,E choices 
both pre-instructional and post-' instructional Next to 
each .test Item number there Is Indicated the" number ,ol . 
times a student made ■'One of ttie twelve possible choices - 

Th I s repp^ i s ex treme ly important " I n ya T i dat i ng the 
test items; and improving the, quality of distractors. In 

, . '. ■ .o' . ' ' ' ■ ■ , . / ■ . ■ . . , ■ , ' ■ ■• ' ♦ 

the middle of the pa^e, test item 2080500 102, ^^ twelve 
Students Chose' answer A pre-instructional ly and 11 students 
chose answer D pre-inst rue t Tonally. This is usually a 
good indication that there is a problem with the distractor 
or the right answer. It Is a good signal that this item 
requires re-writing. Similar pa'tterns^'can reveal problems 
with test items. Constant refinement of the test items 

villi result, eventually^ in a set of test items which are 

^ ■ • ' ' - ' ,. 

tai lor-made and standardized for the local district. 
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Computer Procedure * . ' * 

!• Brentwood has an NCR Century 200. The cbmputer has 

32K mernory» three 657 high stack disc drfvM, Companldh. 
equipment Irrcludes a high speed printer, card deader and 
Op-Scan 100 mark reader* ^ " 

2. All banks of information are originated In the currrculum 
off ice which optional ly submits test Items coded on the 
data processing forms. - ^ 

' 3- All forms are keypunched arid card listings are senf back 
to the curriculum off ice for refinelnent or confirmation. 
Any corrections are rekeypunched and a new 9ard listing 
Is generated. This process continues until th6 card 
listing Is confirmed correct, 

k. Objectives (and any items) are then stored in computer 
accessible banks. The LIST routine generates a district 
copy of the bank. As currlcul^jms are reflped, objectives 

' ^ can be added, deleted or , changed. in the computer banks by 
using the ADD, DELETE or CHANGE function respectively. 
The curriculum office submits the appropriate function, 
forms, and the process cycles *as in step 3 until accuracy 
confirmation is received. - 

S. Curriculum Office submits Continuous Trend Test construction 
forms* Process cycles as in step 3 until accuracy con- 
firmation is received. - 
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6, Th6 computer generates Continuous Trend Tests for the , 



I ^ semester. TQhese are sent to the district for dlstrl-* * 
but Ion. 

1. ' Ansvier k^ys, for the semester are then put up In the com- 

putlr banks. They are either automatically generated 

'* - ^' ■ ■■ . ■ . ■;' ' ■ • / ■ ■ 

from optional master ansiwer k^y files, or el$e the cur** 

: riculum office must submit Answer Key Forms. In the second 
' case, feh^i- prpces^^^ cycles as In step 3 unt l| accuracy con- 
-firmatlon Is received. In either case, answ^ keys are 
estcbllshed and a listing Is automat ical ly generated for 
the district's use* . 
^.8.' A student data base is established on the compyter either, 
from an existing base or from Input forms.' If the base Is 
established from Input for;ms, t^he^process cycles as in 
; > step 3. In either qasej when^the student data bgse is es- 
tablished the LIST program will generate a district, copy. 1 
9. After the student data base is ^stabl Ished, the computer 
^enejfates a semester's set of Test Schedule^^ for each 
- teacher. These are sent to the district for distritjut ion 
and Verificationi " ^ 

1^.* When Test Schedules have been verified, each teacher re- 
ceiyes a semester's set of preprinted student response 
sheets. * \ 

.11. Teachers submit any student file function requests a week 
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before each test. f : 

12* Forms are keypunched and a card 1 j stf ng f s made for^ 

verification. Any corrections are rekeypunched and a new 
• listing Is sent, Thl^ proces^s continues until the llstlng 

Is confirmed correct, • 
13^ Stuc^^t file maintenance is then processed. The LIST 

program generates a list tng of the refined ^^^^^^^ 

base. This Is sent back for distribution. ' • 
1^*. Teachers administer tests.* . • 

15. Marksense forms are converted to Op-Scan cards. A 
sample of 5 put of each TOO eards will be checked to 
guard against hardware fal lures, on the Op-Scan machine. 

16. Tests are processed and student/ teach^er/content reports 

r are generated for .distribution (15 hour turn-^a-round time.) 

During the summer of 1972 Public Systems Researcb^prov I ded 
consultation services for programming the Brentwood computer, the 
program. was In Fortran IV and adequately served 2000 students In 
4th, 5th and 6th gra^e mathematics. ' However, the cyrrlculum officer 
reading consultants and teachers had al ready selected object Ives 
and test i^ems to begin Reading Comprehensive Achievement Monitoring 

Difficulties with Fortran IV In the NCR Century 200 precluded 
any "Expansion. In th^ first half of 1973 the Coordinator of Data 
Processing, Mr. Joseph Rotolo,/undertook to reprogram CAM In COBOL.. 
The results surpassed expectations in the following way: 

• - ' - . ■ . ''-^ • . ; • - ■ ■ 
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if ' 

•■*•»' 




\ . ] 


■ V ' 


turn^around tinie reduced ' ^ ^ 






b;' capab.I l I ty expanded to handle up to four programs for 


^ ' i 




' 22000 students ' 


■ 7 ■ 


-■; ■■- ■ ■ ■ • ■• ■ ■ ■ ' 


c. increased accuracy 


■■ *' 




We now process tests at 9< (vs 25* at Board of ^Cooperative _ , . 




-■ " ■ • ' 


Educational Services) per student, TJils Is basical ly an expenditure ^ ^ 






for paper since al 1 equipment and al I; jsalaries were being paid for 






.business ar^d Clerical ftinctions^ The fact that we now prpcess 






>^ more than 500A ^nswe^ every test period without adding to 






costs Indicates^hat personnel and equipment were previously under 
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utMized, / ' . 

■ j ■,.*/'■. 
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IV* Support Material's Developed for rthe Mbnitorinq Program, 


- y., 


.- ■ ■ ■ ■ 


■ . •■■ ■.■.r- ■ • 
The most pbvlous need for support materials was for a publf- 


kj 




cation that would open the Instructfonal program for everyone in 


■ 




^ the school district. A committee of administratdfs and teachers 






was ^rmed early In the program to produce such q booklet. The 
B^E.S.T, Manual for Students and Parents (see appendix B) was the 






putcome^ The booklet attempted the fpl lowing: - 


J 

-'■ ■ - -'A 




!• To explain the Instructional program before the student 


.f' . • ■ ' ' 'i 




receives an evaluation. 




" . / 


2. To\ex|5N|a|n that the student vii 1 1 be tested regularly and 


- 1 




- ^ ■ ' • ■ ... 

that reports will be sent home. 






3* To list the minimal' set of objectives In mathematics for ^^|| 


..1 




eack level . 


" . ''• * 


. # 


To supply sample test items * 






a. .to give an indication on how the students knowledge ^ 






and skill would be tested ' * ♦ 






b* to 'help make the objective clear by providing an ex- 






ample. " ^ • . ^ 




• 


. This publication wSs distributed to students and.parents at 


* ■'■ 1 


• 


the beginning of the school year ♦ It was general ly .well accept-ecl 
except for a. few parents who thought that the Jnstruct lonal program 

was"|He school 's business and should remain that way. Many parents 

*■ ' ■ • • * * ' 

did not react. Those who did psually had praise for the_ booklet 






and the communication about the instructional prog/am. 
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• For further support df the monitoring program, the BEST 

■ ■ . > ■ 




■• -. ■ *■ ■ ' 


Committee decided that tt was necessary to examine the mateHals • - 






"^avatlable to Breihtwopd teachers and to cross-reference the math 




• ■ %•• 

.* * f 


objectives to those majzerlals- (see appendix E) A Cross Reference 

. - . », ' . ■ - . ^ ■ , • 

Guide was developed for each grade A, 5 and 6> 




f ^ : 

■ . \ • •■ 


The adopted math books were the Houghton Mlffl In Series, We 






^ ' ' ' V - . '■ ^ ' ' ' ' ■ ' ' ■ ^. ' • ■>*•■ ' 
had on hand, some new texts, some old texts (which v/ere left-over 

■ ■' - " ■ ■■ ■ 


\ 




from before the Houghton Miff 1 In revlslpn) and the-Addlson WeS^ 


*» 




Alternatives set of texts (made available to schools- and, teachers 


J* ■ 




. reluctant to adopt the HoUgtiton Mif f 1 In:) » These texts and the . ^ 






audiovisual materials which supported \theHbughtonM 




■ . ■ . ' s 


constituted nearly the total aval lable materials for teaching math 




• . 


In" Brentwood., When the objectives had heen' pubi.l lshed In the Manual 
for Students and Parents, It seemed necessary to Indicate where 
these objectives were taught; where could be found drill and practlcp 






to help sharpen student skills. ; ^ ^ 






- Level by lev^l the objectives were matched to the materials 






(see fig. 18). This hookJet now made easy a new approach In which 




* 


teachers could now start withan objective Instead* of following 






. text book sequence. Anticipating that teachers In reviewing the 

i ■ • 






tests would -request a handy and usable answer key, It was decided 






••■ ^ '■ 

to make a booklet which provided ea'^ch teacher with a complete set 

of answers for each te$,t item In the program, (see f ig. 19, appen- 






' dix F.) The objective ^is listed In the column. To the right of 
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Cross Reference Guide listlrvg 
Instnuctlonal materials by * 
objectives 
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BRENiUWOOD E.0UCATION SYSTEM FOR .TESTIITG 

GRADE FOUR 
*■ v ANSWER KEY T?-T^^^ ' 

2na. SEMESTER 
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Answer Key 



ft ' f. 



the objective Is llste'd t\}e corredt answer for each of the test 
Items developed for that .Objective, " 

- ^ ' ^ '. : . . . V ■ ■ ■ ' . 

During the first year qpe of the BEST. teaifiMeaders felt a 

* ■ ' . ■ I. ■ ■ ' ■ 

need ^or the development of practice activities for each of the 

objectives in tl\e 6th gride curriculum. Working entirely on his 

/-'■.■■.-'", 

own, Mr. W-ill/Iam Harris of the Northwest Elementary School, wrote 
his book. of supplementary exercises* (sear*AppendIx J) The Cur- 
^^iculum office was sufficiently Impreissed with this activity book 
that it was *pj:jnted and distributed Jto all 6th gl^de t,eachers. On 
a visit to our project a member of the State Educatiofi Department 
askoid for some copies* Eventually It was' duplicated and distrilyjted 

Itftroughput the State *of New * York. ^ * / 

' ■ ■ * ■ " ^ ' . . . • * 

Also at the^ompletion of thjfe first year, the corranljttee of 
teachers and administrators begarj to compile mastery tests (see 
Appendix O) for each objective. It was a conscious, deliberate 
step in the direction; of mastery teaching. For eacK level in our 
Instructional program a set of minimal objectives* was defined by 
teachers and administrators. However , even at the reading of the 
list, it is clear that some objectives require ^nastery tn the 6th . 
grade. For example - #6210 - The students wllj select the number* 
which i? the correct answer to an addi..^ion problem consisting of 
nd^ fnore* than four^5"digit numbers. This objective should be ac- 
complished a< a high mastery level by most M:h.^raders. Normal 
6th grade students. Should get the correct answer nine out of ten 
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times. * * ^ 




« 


On the other hand, Objective #6130 -"Student will select a/ 






ba$e Six number which Is the same as a given two-digit base ten 


■ • * . • \ 




number or will select^, a. base ten number which Is the sam^ as a 






given three-digit base six number." Is the kind of math problem 




V 


and thinking that all students should be exposed to. We know that - 




V - # . . 


this objective Wll 1 not ^e mastered "(with retention) by all students. 


\ 




To determine the level of mastery, tests consist I ng of many 




C 


. Items for one objective were constructed. (see fig. 20) The uses bf 






-tjhe mastery test were then dievelpped: 






1. to provide Information beyond the regular trend test. 






2. to check or validate results on trend tests. 




1 


,3# to determine the regularity with which a student can suc- 






jcessful ly perform , the action called for In a particular 






behavioral objective. 






Iri addition a variety of charts, graphs and''' forms have been 






•developed by Individual teachers. The BEST Objectives Chart (s^e 






fig. i1) was developed by one teacher and requested by many others 




• 


for making visible the class record against the set of objectives. 






This chart may be used In either the math or reading program. It 






lists students and lists objectives. C = Corrrect - W Wrong - ^ 






• * 
ft «='No Response and T = Taught. Since all the information of this 






chart Is produced by the computer for the Individual Analysis report. 






It seems that there Is some other benefit derived from displaying 






the same results on a wall size chart. - 
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01-03^01 



STUDENT N/y^ 



OaJECTIVE 

TEACHER fWME ° . . . / ' ' DATE 



'V 



■1 . 

li' 


-M.,.,^ ^ ^ P — ^ — ^ -~ — 








1, 


VAi/VT^ IS Wt equiVAi^ to 6/12? 

^ ■ ^ • ■ • . • . . ' . , ■ 




\ ■ • . 

* .• o' ^ 


■ o ' 
■ A 


CA) 


1/2 CB) V18 CO 3/6 

. >-p^^ — — '- — • C — ' '■ — ■ ' 


CO)" 


12/24 




2. . 


WHAT FRAptlON- IS NOT EQUIVALENT TO ^1/12? 








CA) 


3/9 . ' " CB) m- ^^^^^^^ 2/9 ' 


CD) 


6/18 


i ' i 


3., 


WHAT FRACTION IS NOT EQUIVALEfeJt TO 6/l6? 




■ • 




CA) 


3/8 J Cb5 21/56 CO 12/3^- 


. CD) 


6/18 . 


t 




WflAT, FRACTION IS NOT EjQUIVALENT TO 10/20? 






- V ' 


CA) 


5/9 CB) m CO '8A16 


* CD) 


5/10 




5. 


' WfiJ FRACflON- IS NOT EQUIVALENT TO 6/8? 










9/10 ."■ Cb:^3/^ ] . • CO 15/20 

V. ■ ' V 

* ■ t ■ 

\ - ' V '. ' 


CD) 


21/28 






' " .'v ■ " ■ " 










. V . » • ^ ■ 

; • ; ' ; ' ,(fig. 20) 


Mastery Test . 
sample page • 
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(fig. 21). , . 
Class Performance Chart 
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V, How It Works I Once organized the CAM System or the Brentwood 
yarlat lof? B.E.S.T. , provides the followkig: 

I. A curriculum defined by behavioral objectives/ which are 

isystematically coded for identification, retrieval and 

• > . • • • ■ . . ' ■ ' * 

" ^grouping, and which are further classified by one or more 

^ • - *^ . ■ ■■ " ■ 

taxonomies. . „ . - / , ^ 

. 2, Test Items or other behavioral inclica tors designed to' 

measure studejit performance on each behavioral objective' 

specified forthe system* 

3* A set of -randomly interchangeable tests, organized so 

■ ' ■ ' . ' . • ' ■ ♦» 

that each test evaluates all or a predeternrined sample 

' ' ■ ' f- ' 

^ of all the objectives in the curriculum. * 

A. Periodic tggting, usually weekly or biweekly, t,hrougho.t?t 



the period of th6 course. ^ 



5. Computerized analysis and reporting of results within a 
#ew days of each testing. 

6. Interpretation of results^ b^^eachiers/ and students as a 
basis for decisions relating to curricula, instruction, 

, and estimating ofl^ student progress,^ * 

7. Modification of curricula,' instruct ional activities and 
the CAM design based on the results extending over the 
.course. . ^ 
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8, Reporting of student progress.^ 

In the Brentwood System, ^the first test In the twelve test, 
sequence is used asVpretest. This system provides the added ad- 
vantage of having all results stored In the computer so that the 
very next test provides information concerning growth, retention 
and new learning. . ^ , 

As the course progressed, the monitoring system serves to 

' ' . ' ' " 

provide feedback to student, teacher ^nd parent concerning progress. 

' ' .y ■ ■ ■ - -■ • 

It might schematically be presented as follows: 



4) 
o 

Q. 
O. 

<u as 

JC c 
SI 

u u 
o 



u 

JQ 
<U 



Objectives 



Curriculum Activities 
for those objectives 



BEST Testjs to Monitor 
Progress , Retent i on 



■ k 



Computer Generated Reports 



Instructional Nodule 3200, How Does CAM Relate to Course 
Structure , WilKfam Gorjth and Robert O'P.elMy, SPPEO, -New 
York State Education Department ,1972. • , - 



. f 

* ' '■ ■ 
*' * . ■ 


' ' ' ■* ' ■ ■ ^ ' ' * ■ * 

, " _ , . • , , 


.I'i 




^ . ..... i . . ^ . . ' . 

■■■ i<t . . ■ ■ . 
The relationship of the monTtorfng system to the Instructional 




f 

4 

• 


t)rogram Is dynamic. The constant f lovf of Information becomes part 
of the curriculum. It provides a problem detection and self cor- 
- rectlon mechanism' for the teacher and the course. 

t ■ ' ^ 

■ •■ "• ■• . ■ ■ ■ . t 

, ■ ■ - ^ (r ■ ' ' ■ 

. \ . \ 

. •■ ' • ' \ * - 

■ ^ . • .... 

.. ■■' ■ •*' . ' ■ • ' 

% ' ' " . ■ f ' 

: ' ^ i r ... 1 
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VI* Using Honltorl/ng Program Information for makinci Decisions 
about the I nstructional Programs TeacherSt principals, 



students and ci^rrlculum directors have to make a long series of 
educational d^dlslons during the^course of a school year. In 

most cases these decisions are built on Intuition, past practice, 

1 * 

habit,, astrology, superstition and other less scientifically sound 
methods. Asking the f-lght kinds of questions becomes a^-major ' 
pursuit for thos0 cQr.ficulum directors, principals, etc^who have 
developed an information system to monitor the student's progress. 
Some regular questions are: ' * % 

1. How-tiong does It take? 

2. How* effective was past Jt>st ruction? 

' *^ ■ ' i ' ■ . • 

3. ,How do materials compare for achieving" obtjrect Ives? ^ 

How do teaching "methods compare for achieving objectives? 
5. Which sequence Is mostV effective in achieving objectives? 
6.. Whjch test. practices are most helpful in assessing progress 

and retention? 

Information from the monitoring system will protyide feedback about: 
1. Individuals • 
2i groups ^ 

3. methods-activities > 
k\ materials ^ ^ 

5. times 

. 6. the testing system Itself,/ ^ . 



After schools decide what they should teach^ a monitoring 
system serves ;Several purposes. ^ . ' 

a. It keeps track of how well students are learning and 
remembering. 

b. It provides information for specifying the curriculum. 

c. It provides information for comparing effectiveness of - 
^Ifchdds, materials, groupings, etc, on student progress 
toward achievement of goa1^» 

\ ' d. Itspcovides Information .for faculty discussion of problems 
of instruction.. It m^kes team work more purposeful, ef- 
ficient and productive. ' 
I e. It focusics the instructional urogram by regularly calling 
i * to the minds of students, parents and teachers the academic 

■ r • . ■ ' " ' ^ ' ' 

1 - behaviors that are being sought and developed, . 

Information reinforces the professional nature of the te?cher*s 
and the schools task/ From the. three years experience with BEST 
in, Brentwood, a teacher survey (See Fournler Midi, May 1975) clearly 
indicates that teachers consider the Information In choosing, the . 
materials, means and methods of Instruction, 'they frequently and 
regularly report that while the system* makes evaluation dnd assess*- 
ment of student progress easier. It requires the teachers to make^ 
more and more decisions. , 

A constant flow of decisions about individual students must 

0 . * 

be made: there must be an assessment of progress toward each ob- 



* ■ 

■ ■ • 


\ ■ . ■■ . ■ ■ ' V' - ;v ' . • ' • ■. ' - ^ ■ 

> ■ ■. •■ V ' i . . • . • 

I ^ ■ ■ .• ■ ■ ■ ■ 












jectlve; there must be determined priority, ctcdier for selecting 






objectives for the Individual to work onj; there must be establ ished 






pattern of organizing the class to arrange for opportunity to an 


• 




Individual to work on his own problems. • 




• 


Decisions about group progress are based on " 






a. average of percent scores on total tests ' 






^ b. group percent on each specific objective; \ 






^ The general curriculum decision^ ^bput student progress, ' 


•' .} 




whether examining Individual or group achtevemtfnt usually Is hiade 






In one or more of four^sual alternatives: 






- • av ^ The objective may have been too difficult, irrelevant and. 






shoulct be eliminated. 






b.. Information appears to be Insufficient, calling for the 


■ ** ^ 




addition of objeciiives to be taught and tested. 






c. Timing is Inappropriate. That Is, that the objective 


" ^ .1 




Should be rescheduled to appear In some other part of 


\ - 




* the sequence of curriculum, • ' \. ' 






d. The tests themselves s^re faulty and Items should.be 






rewritten or replaced. 






The Information that provides a basis for decision flow 


■ i 




naturally from a criterion referenced evaluation system. The In*- 






formatlon produced by each C.R.TI should Include the following 






minimum: ^ . 


* 




a. did the student learn the objectives just taught? 




^ ■ < ■ 
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4 


b. which objectives will be taught later - which should be 




m 


pretested? fto determine vjho needs Instructions and who 




m 


doesn' t.^ 






c. what does the student remember 'from objectives taught 


* : 




earlier In the course. That Is, what is his retention? ^ 




t •, ■ > 


From this Information a teacher and student can proceed to ' 




r 


examine the Reports. The student's report (see fig. 10) gives In 






' graphic detail his performance on a set of objectives- that had been- 


» 




specified.' In the di'scusslon with the teacher and examination of 






his own the student gains Information which \ ' 






a. clarifies objectives only working on them gives a ' * 




A 


nearly complete account iDf the behavior expected. 






. b. immedtately reinforced behavior which brought about success , 






for an objective. The teacher ''S awareness of the behavior 


\ 




and the success or failure, make It much easier to decide . 






about rewarding, re-lnforcing or helping the ^student to 






; 'avoid specific behaviors; In many ways, >^ls act! vijiy Is 






/\ like correcting teacher-made tests. However, there Is a 






. great difference in the amount and form of reporting pre- 


..J 




Instructional , post-^instruct ional and retention behavior. 






lawmaking decisions; for group or whole class 1iis,t ruction: 


♦ 




"How does one go abo*ut doing this? By methodical ly analyzing 






and interpreting CAM feedback data.' The'method used involves: 






(1) detecting logical* indicators of progress or a lack of 
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progress, (2) llstfng the possible causes . (3) Vlstlng the 
cone 1 us tons, (l,e*, most probable c^e),. (^) listing alter- 
native solutions and, (5) making a dec! sfor> concerning what 
must be done In order to benefit the student— the tncllvldual.^- 
to help him to learn better (i.e., se]e.otIng the best alter- 
native). This applies to "fast!' students as well as the 
slower ones* 

If anyone should ask you' where,. In CAM feedback, one. 
should look for Indlfcators that all Is or Is not going weH,* 
you might answer,^ 'Everywhere. * * , 

^Everywhere', fn this case, I ncludes dur Ing . the .entire 
pre- Instruction, phase (pretesting) Immediately aft^r in- 
struction and In the entire post-instruct lort phase . 

What are some of the things that cap happen during these 
.phases? Wei 1 here are a few. During the pre-lnstructlon 
phase you. may find that the student either already knew the 

^- ' - ■ ■ . ... . 

subject or' that he learned aU or some jof It during that phase. 
A h^igh pretest score .Indicates one or mor^.of theie possi^bM- 
Itles. . ^ > ' ' V 

A low score In the Immediate post^-Instructlon phase In- 
dicates that either some deficiency exists In the student's 
learning or the efficiency of ' the Instr'uct Ion or t>oth. And, 
a decreasing post test score indicates diminishing retention. 
There are many other i ndlcators* 





* ■ '> ■■ . .. ^ % , . . . . ^ . 
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Why, any or all of these situations came about Is, part of 


« 




the detection—analysis — process. What to do about It Is th^ 






decision." ^' ^ ' 

• ■• • ■ ■ . ■■ ' ■// 






^ <■ \ ' ' '. ' 
ft ■ . ■ ' ■• ■ " ■ . , 
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^ SPPED» Instructional Module 3000, New Yofk State Education Dept.', . 
1972., William Gorth, Richard Al len, Ro"ber..t O'P.eilly, 
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VH. Mastery Testing Component of JEST . Rastery' tests/ *($ee Ap- ' 
pehdlx D) are a necessary extension of the trend monTtorlng. Trend 
monltorfng tests on f^ew (maybe one) Items per objective. - Mdstery 
•Testing, tests many items on an objective (5-100 or more). Mastery 
Testlng a? indicated by its -name tiells a lot about a little bijt 
Cone objective). The uses of Mastery Testing are:. 

for additional information - student performance on trend 

tests does not satisfy teacher that student real ly knows - 

*• J ■ . ' ' ' ^ ■ ' - ' ' ' ■ 

or doesni't; kfiow an objective. 

* • ' ' • .. . ■ V > . 

\ b. for^^conflrmation r Teacher wants to know the extent to 

which the "Student wf 1 1 perform on an objective. Trend 

■■ ' ■ ^ ■ ■ ■ . " ■ ^ . . 

data wil 1 give^one oi/t of one test item correct or ln~ 

correct for any objective. . 

Mastery test will report 6/8 or 8/8 or 0/8 correct.. The' 
teacher gains added assurance about the lever*of student performarnce 
and the rel iabi li ty of his performance on a particular objective^ 

In Brentwood, M^dStery testes a/e in the s|:brage rooms or media ' 

centers of each elementary scHool . A lEea.cher aide on demand will 

run off a ditto of a mastery test. Teacher will administer the * . 

test. The aide may correct the test, using the answer" key. the ' 

results are immediately forwarded to the teacher tor instructT'onai 

- 

decisions based upon both trend and mastery data* ' . 
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Implementation of the BEST Program tn Brentwood ♦ In 1971 

Or^ Robert 0*Reil1y^ Chief of the Burejau of Research and Cultural^ 

Affairs, New York 5tate Education and D<f. William Gorth of %he 

University of Massachussetts came to Brentwood for a dinner meeting 

with Or. Artrhu'r R. Bci^ger, Assistant Superintendent and with me, 

Director of Curriculum K-12. They had a syi^t-em they/ were trying'to 

sell the Brentwood Schools and they were aware' of many advantages 

of working in Brentwood. Firsty^ the^ size of the school, district. 

^ ■ t> . ■' • • ' ' ■ ' 

With 22,000 Students^ at that time, Brentwood f-anked as largist 
after the big six cities* .Secondly, the B r en tv/obd Schools bad de- 
veloped*a reputation for inftova^tion in curriculum*^ A six-year Ford 
Foundation School Improvement Grant (which ihis writer directed in 
1966-1967 and 1968) had resulted in the'^publ icat ior> of several 
programs which were di^stribut^d natiqnally. 

The CAM system they ^ described seemed hopelessly complex at 
that first meeting, yet, there was a great-appeal in the system- 
atization of 'information that was offered. Dr. Brieger and I were 
aware that our scores in reading ancl math iiad been decl ining .at a 
rate greater than the state average. We had also become caught in 
budget squeezes which forced the eTwin nation of Supervisory p^sitlonSi 
Our class size was (and is) the largest iri the area.^ Even before 
wej^ere fully aware of the understanding of the potential of 'the 
program, there was an unwritten understanding that if we pursued 
pur interest at all, it- would be with the committment of very little 
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mofteyi. We were assured that if we qualified there would be flnan- . 

cfal support from the State Education Department and from various 

* ■ . ■• 

government sources. ^ 

I had been Curriculum Coordinator since ]S6Bu In those 
earlier years, we had coordinators of ^Social Stfcdles, Reading, 
Science, etc. Since those positions had been lost and we stilt 
had twenty schools', 22,000 studentsand Important curriculum 
problems, mV first tncl Inat Ion was. to try to find someone else to 
pursue the Information given to us by Dr, Gorth and Dr. O'Reilly. 
I went to Albany for further talks with Or. O'Reilly and he provided 
me with various descriptions of the system. Slowly the potential 
for monitoring Instruction became clearer and clearer. Though I'tv 
still seemed difficult to find the time, as I grew to know the 
program better it became evident to me that ultimately this project 
was definitely part of my responsibility, and further, an oppor- 
tunity to improve the instructional program. Dr. O'Reilly's 
definition of Curriculum as "the set of behavioral objectives" at 
first seemed. an overstatement and yet there -was enough of a "truth- 
ring" to the statement that It discouraged .my assigning such fund- 
amental curriculum work^to some other d'dministrators. For several 
weeks I studied the program. 'I even trled'explalning ft to other 
people without. any success at all. This is when It first became 
evide^ht that what Is really an incfedl'bly simple concept Ioi> becomes 
totally tangled in explanation so that the uninitiated usual ly re; 



'>■•'; ■ » .A ". . 

•,"**." «• . • > ' 
»'■" 


< ■ • • \ ^ ■ .' • ■ . . . ' ' • ■ ' 0 •. 
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< ■ . ■ » ■ ^ ' ^ " ■ . ' •■ 

^ ■ ' : ^ -^^ - ■• * , •.. ■ • . ■ > .m.. 






' ^ > — — [['. •■ ■ '■' . ^ ■' , . ' - ' . — -^^ — . ■ .i. 

celve. a' itiessage? that says somethlng^^ 1 ^ 'There are' so many . 


" " *' . i 




var labiesi" object Ive^s, coniput<5.rs> numbers, -etc, that . 1;' 11 never 






i TeaMy-- understand this.'" 't*^ was a great relief to discover that 




^<*^ ■ 


- only thk project manager had to understand al T the educat lpnal . Im--^ 
pUcatldns and only the compufter- programmer had to understand ajl 


■ •' * \ 




thi^'domputer fmpl Icat Ions*. It^ was entirely possible for people to 




> ■ . ^* - 


6se the system without understand ing the'relat lopshlps. 
- Dr. O'Reilly was' good enough to arrange fof a visit of Brentwood > 
personni^l to two New York State school districts which were At- 
tempting the prog ram". / * 
ChWsflng tl^e Ind^lvlcfeals to go on this trip was 4:he subject of 
long discus^sjfohs between the curriculum coordinator and* the assfstant 


.9' ■ '] 
i« . • t > 

. * 




superintendent. The people selected for the- trip w,ere l ikely .to be 






the people who would try the systenrjf- we ever, procee'ded to that 


j 




, point." '^^^ Immediately agreed that all thosfe people who .would 






accompany mtl- vfould^ be principals* , o . * 

In selecting three principals frohi among fourteen,' certain . 




to 

■. • » » * * 

«■" 


criteria had to be followed.. ^ ; 

" - ** . . . . ■ ■' ' . /■ 


^ 1 




^ -v; 1. . They shbuW be principals who had shown a will ingness to . 






•. . innovate.. ^ r - . • 


,-3 


. ' • ■ ■ ^ ■ . 


2* They shjsy.Td be opinion leaders who cpuld influence others f 


• - .N, . ■ - .' 


• • ft . " 

■ . . * '. • 


. ^ to' attempt^ the ibrog ram, should" It succeed. 






^ 3. They.haxl to.be [jrhncfpals from among our six principals 






^ • of target ^schools --the target sjchool s being, defined* a^s 

* . . ^ . -^^ ■'***' ■ • * ' . * ■ 'i ■ ■ ^ . ' ■ ^ 
■ .* * > ■ •■*•■- c V- 
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those with students from famll les. receiving aid to 

dependent children,* ^ . 

: If we were %to spend any money at ^11 on such a project I t 

would certainly have to ccxne frc!4i ESEA Ti tie .I.<funds from the 

Federal Government. ' Given all the needs ana restrictions/ three 

were finally chosen. The first choice-had' been a math special 1st . 

for training teachers when we had Ford Foundation money for^ modern 

math, the second was a lon§ esta.bl ished principal who was a - ' 

known opinion leader.' The tlliwd was hew to oup district but with 

many years of experience in other districts. He often had demon^ 

strated; a wi 1 1 ingnes.s^ to innovate. Each was approached. 'Th§ 

program was expla.Ined.but fther^^ was sudden surge of enthusiasm^ for 

the idea of a mon I^tor I ng program. However, they agreed tcgo to 

Ballston spa and to Greece, ;'New York.' ^ ' 

On'this trip,. 'the decision to go ahead and attempt the program 

was really mSde, though It took Board of Education action at a 

later time to make It official, the ^fi rst school was visited - 

Ballston Spa Middle School • would have been difficult td fln,d 

anything, that resembled our own school district less. Mt was a 

very large, modern, expensive building to house«grades 6, ^ and 8; 

It was constructed for large group, small group and I ndi vidua 11 zed 
• • ' • ' ' * * .■ . • . ^ • * 

Ijistruction in open space'. The various ^aijeas were brightly decor- - 

ated./*The staffing patterns* were even^more rembved from our ex- 

.perlence. A team leader, or grade le'ader worked with other Iriernbers \ 
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of the staff to develop objectives in each subject. Tests were 


I* 




constructed. Upon completion of units students were directed T 






to the test area where tests^were administered and scored by aids. 






CAM tests were sent to the Data Processing Center which produced. 


■ • 


; . ■ .: . 


the reports for each individual teacher. Everything seemed to be 






•1 ' . ■ 
working very wefl and yet the impact on our group from* Brentwood 




^- " 


was negative. Everything was so new and expensive; so much man- 

/■ _ ■ . ■ 


: 


.; <^ ' 1 


/ power was available that it seemed to us that these njust be the . ' : 

„ • • ft ■ , • ' " 






factors that made CAM "go" in this schoo^^istr let* It was for- 






tunate that* Dr. O'Reilly had arranged for us to visit Gr^ce^ New 






YoriC as well. In Greece, we still found a better personnel sit- . 




} 


uation than we could have. Yet the project seemed to be getting) 


* .' 




Started in this district which in many ways was not too far re- 






, moved from our reality. The teachers and administrators' we tal4<ed 






to were very positive about this program for improving student 






, perfdrnance.. On the_way home, we discussed how we go about giv4fig 


■V 




it a try in Brentwood. 


> 




The natural step was 'to get permission to have some people 






trained in CompVehens ive Achievement Monitoring. It seemed natural 






and necessary to tnclMde teachers jn each of the three buildings 






where the program was to be tried. Just "as naturally^e thought 






• ■ ■ . • ■ ■ K . '- r' ' 
of those three teachers as ifrbe t^am leaders-to help initiate It 

. . ■ i , ■ . 






in each of the schools. The same pi?e/equisitfes iseemed to apply to 






the^ teadhers^-who would, help this program succeed. They needed to 






.be innovative and to be opinion leaders in ea.ch of their buildings. 
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' Once the teachers were selected, all six people went to a one week 
workshop at the University of Massachussetts* Dr. William Gorth 
conducted Intensive, all day programs which. described, demonstrated 
and simulja'ted the CAM operation. Our three principals and three 
teachers returnea to Brentwood as authorities In monitoring In- 
^structlon. At this point we had enthusigsin and kngwledge and 
J nothing else._Yet, we decided to Implement the program. .. In , 
order to Implement the program^ we would need objectives, test 
items, tests, guides, answer sheets, etc. etc.. To get organized 
^ for producing and later carrying out these tasks we descRibed^the" 
"following organization, (see fig. 22) * 

It was decided that the Di rector of CTurrlculum would be. the 
project manager. The director would report to the assistant 
superintendent who tn turn would report all phases except evaluation 

to tlje superintendent. The administrator for evalliation (at tiiat 

♦ -» 

^ tipie <5n Staff for ESEA) was to report evaluation ^directly to th% 
superintend'ent. The director would assume responsibility for all 
. relationV: Internally with principals, team leaders, central office, 
student$,\lstrlct curriculum committees, Data Processing Center and 
the State Education Department*.' , • ,r 

At this time we also carefully defined the team leader's role, 
(see fig. 23) The team leader would carry on dii^ctl iaison be- 
tween faculty and principal. He would cop^luct ' in-service workshop5s 
for interpretation of data and explanation of process. 



PROJECT DIRECTOR'S ROLE 



PRINCIPALS 
TEAM LEAJ3ERS 

FACULTY 4-^ STUDENTS 



DATA 
PROCESS ING- 
" CENTER 



SUPERINTENDENT <■ 

■A 

ASSISTANT 
-SUPERINTENDENT 



EVALUATOR 




STATE. EDUCATION 
DEPARTI«1ENT ' 



> DISTRICT mVt\ 
COM MITTEE 



DISTRICT READING 
COiYT'lITTEE 



ADDITIOMAL DIRECTOR DUTIES! 



1.,- SCHEDULES F©R TESTII-JG, REPORTING. 

2. CaCLHCT f^lEETINGS - AGE^DA, MINUTES. 

3. REPORT TO ESOARD OF EDUCATION AND PUBLIC. 
K. ASSIST SCHOOLS 

A. IN SERVICE^FOR PACULTY . 

B. .ORIENTATION FOR PARENTS " 

5. SUFBINMSE PRODUCTION TO QUALITY CONTROL 

A. OBJECTIVE 

B. ITEMS 
ri C. TESTS 



(fig. 22) 
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Chart 1 1 1 ustrati ng relationships 
for proj ect d i rector 



TEAM LEADER ROLE 



SU^INTEM5B^ - ASS'T SUPER Il^fTEMDEI^ 



DlRECrtR 



DAFA PROCESSIMG 
CENTER 




TEAM LEADER 



. STATE EDUCATION 
DEPARTOENT - 




FACULTY 

t 

STUDEl^S 



ROLE! 



PRINCIPAL 



PARENT^ 



1. PROVIDE TESTS> ANS\rER 'SHEETS AND PRINT OUTS FOR TEACHERS AND STUDENTS, 

2. LEAD TEST PERIOD DISCUSSION WITH STAFF. for: . ' 

A. ANALY;SIS OF DATA - , 

B. STRATEGIES FOR CHANGE 

■ C- MATERIALS, PRACTICES ■ , * 

d; o^^•G6I^iG record of deficiences in objectives, itb-iS/ tests 

E. attend f-EETINGS AT district LEVEL TO SI-IARE . concern, materials, . 
PRACTICES 



/ 



(fig/ 23) 
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Chart h lust rating relationships 
for team leaders ^ 



On thefr retMiri from the UnWerslty of Massachusetts/ the 
three teachers > now team leaders and three principals began an 
Intensive effort to produce t;he materia^ needed for .testing in the 
cqming year. We stockpl led a1 1 avai lable banks of objectsives aftd 
test items for them to work with.. The t OX bank; The EHowner's 
Grove, Pa, bank and the \state Education Bank were all very useful* 
However, the work done irt Greece, New York was more helpful' in the • 
initial trial. One of thifi^ early and key decisions made was to test 
^ the existing curriculiim rather than revise i t»^ . To do that, the ^. 

members of the ^eam went thrpugh our text books activity by activity 

• • . \ 

to find the objective that should be implicit In each exercise. 

\ . . - 

Once listed by- grade level theVtask 'became the writing t>f ^t- 

least *five test items of ' equi vai\<5nt difficulty, reliability and 
diff Since a^'great many math tests were aval lable and since 

given a model^fa^th, 5th and 6th grade math test items, similar 
Items can^be produced^^Hareat prqfuVion. This part of the task 
jf)roceede(3j smoothly* Hone l^f tfie^^ireachers v/ho would use this 
monitoring prpgram'had . (to our knowledge) ever heard of such pro- 
grams. Therefore, It became necessary to write a guide for teachers 
to help them use the material. Students and p^arents also had tq 

iearn so a guide would have to be provided for them as well. To- 

. ■■ 

gether with answer keys and tests, all the material would then have 
to be coded for the computer. ' , 

In September, a joint faculty meeting was held for the three 



sdhools who. were going to participate In the program. A team from 
the State Educatfoa Department came to conduct an Introductory, sen- 
tence. About half of the hthf 5th andSth grade teachers attended. 
The attempt to explain the CAM system in an h9ur and a Kal-f.was 
too ambitious. It was^a very warm day Ijn the cafeteria of Northwest 
Elementary School/ Teachers quickly became impatient* with the 
complex systensthat wef? being pres,ented schema t leal IV on acetate 
after acetate. • , ' 

This was to have been the "kick-off", the initial step In an 
ln-§ervice course of about fifteen hours for training teachers In 
ComprehensJ^ve Achievement Monitoring* 

When this session v/a? concluded, I held di'scussions with other 
teachers, principals and team leaders and members of the Bureau of 
Research and School Affairs. With little support* • niade the de- 
cision to omit , eliminate, skip and otherwise avoid the in-service 
education part of the preparation for and Implementation of CAM**. 
This turned out to, be a keyis^de^i^ion for many reasons. It was 
made for the fol lQW|ng reasons.: 



1. I knew there was a national objection to accountability 
develdping in. teacher groups. ^ 

2. I knew that a previous attempt to Implement this pt?ogram 
In another large Long Island Schobl district had been re- 

jected after' extensive in-servLce education. 

■ • • \ ' 

3. J believed (with Socrates atid Dewey to pick some de^l rable 



company) that the teachers would qutckty and almost ef- 
fortlessly learn the system if they would only* administer 
the tests and then work with the students to interpret them. 
This is the way it has developed. We now have hundreds of 
teachers who really are qualified in operating with a mcnltor;^^ 
system. They bdcame qualified by getting and using information in 
a very basic, immediate way. They mlght^fail on an exaft question 
s(bout/^e technique for randomization of student group's or test 
items (which our computer operator has to* kfiow) but tba$ has 
noth^Tig'to do wits-h helping their students learn. Nov/ that the pro- 
gram has been operating successfully for three yearS/wIth hundreds 
of tea^he^^and thousands of students I often look"b>acl< to this as 
one of the most Important decisions' contributing to the program. 
Obtaining Computer Services ; Though: Breiitwood had a computer , 
since 1968, there practical ly was no^rnstructional applicators made. 
It was used for payroll, scheduling, and ^report cards. The Bureau 
of Research made some-contacts to provide us with computer processing 
of -the program. For a charge of 25* per pupil they were to put up ' 
the banks of Information and-generate reports for our schools. At 
several early meetings the Inability of technecians and educators 
to communicate nearly aborted the prograrir. The quality of the ' 
personnel which represented Grumman Data Systems was probably 
intimidating to publ ic/ school people. Some of them had just com- 
pleted work on goverrrment contracts which included the ''nan on %he 
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jnoon*'' project. A great deal of patlence'eventually prevailed..^ A 

set of forms was devised and objectives, items and student In- ' 

* ' • • - ■ . . * ^ , ■» ■ * 

formation were painstakingly coded for^ the computer. One early 

problem tp manifest itstel| was that fef the 20 mile distance betw^^en 

' ' ' ' ^ ■ ' ^ 

Grumman Data Systems ^d Brentwood, The telephone was often In- 

* .V' 

adequate. ^ Much time was. taken by driving. Eventually all of the 
information had been, put up on the computer and prTnt-outs were 

delivered for proof reading*- Here, we began school districts real ^; 

\ , . - 

introdbctlpn to ''quality iontroV^ because even when we proof read 

carefully, mlstakes^vere made in the correction and cddiitg process. 

*■*•.. * , ■ 

«. 

These mistakes, of course, returned In the final tests and student 
lists. But In early October, we were ready to test, we thought. . 
The Fir^t T^st ; In this first trial ther^ was an answer sheet 
provided to each student but the student had J:o print his own name, 
number, teacher name and number and homeroom etc. ^Then he would 
take the test.' Special IBM pencils were required for marking these 
forms. When thfe students had completed the telsts, th6 answer 
sheets were col lecjred and taken to- Grumman <Data Systems for scoring 
and reporting. .We awaited the results which were supposed to cdme 
In hours,^ The' first test took more than a week' and when the* 
results became available we recognl-zed that'many problems had yet 

to be solved. The most important problem was accuracy. Several 

. ' i • ■ ' ' . ■ ■ 

things, Jt seems, have to go wrong when kth, 5th and 6th gr^de 

Students write down information by band « to be copied by key. punch 



♦ 


■ ' • , ■ * ■ . . . " ' o • . ■ ' ' 

■ ■ ■ ' • * ,\ 

— ^ r • : — ^ 






\ ^ * • 
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' ' - Jt 
operators. ' ■ \ 




\ ' , > 


* 1 

a. they make mistakes' 




' ■■ • ■ * 


b. some of them have- handwriting which defies anyone's 




r.^ ■ ' ■ ■ 


reading ski 11 - . 

j~ '*♦• * 




' ' ■ ■ I 


c. some key punch operators make mistakes, 
^ As long as we worked with Grumman Data Systems we never really 
solved this problem. Through careful checking by team leaders and 
aic4, we reduced the numberof errors, 
. We never had enough IBM pencl^ls, Th(?y disappeared at qimazlng 




• 


r^tes. We cut them in half to double the numbers but still couldn't 
•'••fceep uf>, . ^ ' 

^ Heetings were held to try to Improve quality control. It tl;i<l 
improve but never to a satisfactory levels f 

- •■ 1 ■ ■ • ; , ^ - • V, . . • " , ' - V 

^The turnaround problem threatened the program most severely. „ 


K, 




.There is a need fot: immediacy In^ returning, results for teacher; and 


i> ' \ 


' 1 


student's; use. Last week or two weeks ago seems much tdo remote. 


■i 




' ; tesplle. numerous little and big events that were not supposed* 
fp occur^ but did with regularity, the teafchers (and team teaders 
maintained a positive "^attitude about monitoring. An important 

factor may have been that coupled to the complete bmisislon of Inr m^^. 

' . : ' ' * . ' . 
service Irj^truction we had' repeatedly ahnounced^to the teachers 




• 






that the program probably wouldn' t work. We told them that .our 




. ■ . • • # 

» * ■. X 


expectations for the first year were that we would learn tq.glve' ^ 

* - • \ y ' ^ 

the tests and Interpret the results even if those results were npt 

•■ ■ . ^ ' * ■ 
* ' • ■ • ■ i« • ^ • , , 
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'what they should have been. This approach wa? taken t)ecause we 
had had sufficient experience with programs that prom*Isedm&glc 
cures that' we xleclded to minimize expectations so tha't^l^il^real 
prod^uctl vl ty In the progran? would be viewed as a pleasant surprise. 
Describing the possibility of failure and the expectation of much , 
less than total success helped us with .the faculty. They appre- 
ciated candon and realism in educational Innovation* 

Throughout the period the State Education Department main- : . 
tained Its Involvement with us. They had contracted Dr.. Shel ly 
Harrison of Technovatlons (Later - Public Systems Research) and 
Stony Brook University to be a liaison between th^Brentwood Schools 
and Grulnman Data Systems. Members of Dr. O'Reilly's staff visited 
regularly* The Bureau had contracted Dr. S. Alan Cohen (later 
-author of Random House^Hlgh Intensjty Program) to develop objectives, 
and test items in reading. A steady 'Stream of preliminary work-in 
reading kep£ flooding the curriculum office. The -math monitorjng 
program was being praised by studentJf and teachers and It seerted ^ 
desirable to monitor reading 9s well since our scores had beeh-^de- 
dining. Besides, we now had a staff of teacher^ a'hd team leaders 
trained in monitoring system. The systejn w©u,ld* be the same no* 
matter what subject was morfitbred.^ By the Spring^ I had announced 
that two. schools would begin monitoring Behavioral Objec^Tives In 

'reading. The Bureau of Research . had promised an extensive set of^ 

6 • ... 

i>bjectives with many val idated test items for each objective. 



It was, to hear !t described, merely a matter of chooslog thip ^ 
material and Installing the system. 

Also In the Spring, at the conclusion of the first year, 
'Grumman Data Systems announced that Its ra^e would Increase from 
25^ per pupil to jiearly §0^ per pupil per test.' Since we wer^- / - 
planning to test in reading as well, this>ould hav6 meant the 
cost $1.20 per pupil per test period.,' If^Situdents were in . 
both programs. This, was njo re money fhan vJe were .s^jending on'text, 
books and supplies. Sihce we couldn't real|^ affort the twenty five 
cents charge in the first place, the new* cost was absolutely out 

' \ , . • r ,^ . ^ ■ ✓ .- / 

of the question. . 

.*♦*•' * <^ . . 

In discussing this latest problem wi tK Dr. Harrison, heln- 

. . - ^ • •■ • . " , * . / . • . ^ - - ■■ 

^ - . / . 

formed me thBt his prgartizat ion, Technovati^oris, could, for $A,000.00 
prograqj our district computer ' to /perform ^h«*fmoni tori ng tasks at 
muchless^. than 25^ per studen.t. , s . 

oimul taneously-if 1n soap-opera fashion. Dr. Nannlni , the 
Superintendent, announced bis resTgnation effective in June. Of. 
^Nannlni was reluctant to authorize the progriamming Ib^cause he ^ \^ 
thought that such a decision should be made by the new superin- 
tendent. However, If our computer was to be progratjyned by September, 
immediate signing of the contract was necessary. I signed the con- 
tract without authority to do so. My assumption wa% that Improved 
student performance, would ml tigate in my f^or,. If not justify the 

act. Dr.' Harrison and. his crew froq) Jechnovat ions began the work 

/ ■ 

- ' • ■ ■ 
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^^ In th^ summer. From the beginning, Mr; Joseph f^otplo, the school 






distr-?ct's Coordinator of Data Processing said thgt the program In 






> Fortran I T would \ ' / * 


• . \ 




a. tbe too slow ' ^ . v*^. 


i I 


0- 


not allow formal 1 the students that Brentwood would bave * 
. ' to .put In the program If the program were to succeed fn 






both math and read i ng . 






In the meantlnte, Hr. DI Pietro was made Superintendent.-' 1 Knew 






that Pt would generally be his view thaf anyone who committed the 


• 




scbool district to. a $^,000.00 contract wi thout authprJ ty should^be 






^^'^ ' ' . • * • •• ♦ 
, fired — not an altogether unreasonable point of v1ev;. 






To compound the difficult and complex, 1 received a telephone 






* / " \ * ^ ' 
call fromJ)r. O'Reilly informing me that the objectives arjd test 




» 


* '* . * * 
items and financial support would not be available for installing 

the reading program to which 1 had 'committed myself, the. d istr ict\ 




* 




• 


- ' 4 

and the Data Processing Center. If the first year of GAM^math 






testing had gone well, it^s ending promised to be ^n ^cacleniici 






social and personal disaster. # * ' ' . ' . ^ 

Things could hardly have been worse, since there was also a 






recession and a shortage of^obs that Sprlag^ * 


•> 




, Jn thp-Spring of • 1972 the District Reading Committee was ready 






to make its recommendation to the District Curriculum Advisory 
Council - the Curriclilum Coordinator is chairman of this council 




1 


vyhich recommends programs afid materials to the superintendent of ^ 

" \ 

. ^ ,■ . ^ ' ' • • ' 


V --J 
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^ «... 

■ . - .1.^ * ' . ■ . 


1 

T 









' \ \ 






ft , 








schools. This program parried a set of behavioral objectives In 


• 




reading comprehension. Even with -jthls adoption there would not have 


• I 




been sufficient time to write a IV the test Items. However \a strat- 






egy. for extrication began to, form. 1 called, the salesman to my 






' * - ^ • • ' 
office and told him that 1 had heard that, the reading* committed 




; 


^ ... • I 
was goln^ to recommend Jils series for adoption in Brentwood. He 

c 






Indicated that he had h^ard similar rumors . hiinself. , 1 then ex- X 

plained to him that as chairman of the Curriculum Advisory Council 

•* ^ » ■ ^ * 


» 




\ 1 couldn't possibly allovj^uch a recommendation to me heard, much 

Vv . ' \ 






^* t \ 

less sent to the superintendent if the program was not accompanied 




... 


by four te^t Items for e^ach objective. He took the news well and 






recovered quickly. He called his main office. There a vice presl- 


* n 


■ . 


dient called an editor in Denver and within an houij^phere was an 






agreement to provide test Itenjs if the program v;as recommended by 




0 

« 


the Readying Committee, the Curriculum Advisory Council and the 
Superintendent. Now there was a slight hope that the reading 
1 monitoring program^ could be delivered as promised. ' In the meantime. 






the original CAM math team was going to work for -two weeks to 
rewrite objectives and test items which had demonstrated flaws in 
the first year. This was to be a major strength of the program, 
that In a period of time from constant rewriting there woujd emerge 






, a program completely tailored to the o^s^s of our school district. 






As the first b^ 11 from Technovat Tons came^ due, 1 arranged for 






p&ayment with ESEA money that we had planned to use to pay Grunnan 

. * ^ * 
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Data Systems but that amount was now 6>Jhausted and no provision had 
yet been made for the. final payment. 


y 




By the end of the sui^jger the publ Isher, true to his bargalni^- 

supplied four test Items for each objective but they were not \n c 

(} ' - • . ' 








7 . ■ / 


' us^hle form for monitoring program purposes. Their chief defect 
.was that nearly every item required multiple b^ehavfors on the part 






of the students In a panic the last week of the Sfimmer 1 began to 






rewrite the test Items. For the next six weeks 1 wrote every day. 






In jearly September we began coding the math for scoring on ogr own. 






computer. - 






Another bit of rescuing as the Federal' Pnojects Evaluatbf In- 


> 


1 


formed me and the Superintendent that the'students Pn the monitoring 






program- (see evaluation .section) hajd In fact made greater gains than 






the Contro.l ^roup. This publication gave added\ Impetus for going 






on with the. math program. After pubUcation of the results, 1 ' 






* submitted th^ last bill to the Superintendent with^a long explan- 






ation which'^he accepted-. However, many of Mr. Roto l-o^^s pre- * ' ^ 






dictions came through. as we were through the first' semester without 






testing In reading. 




t - * O 


^ The program on our own computer -was slow. We were n^t getting 
the rapid turn around we^had been promised. The program worked 
only with regular attention from the staff of Technpvatlqns. 






Around mid-year^of the second year of CAM (Renqffied B.E.S.T. by . 




• ♦ 


Dr. Harrison as he programmed the conpljlter) -Mr. Rotolo informed us 




a 


- . ; ■/ ^ ■ / . .. 
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that If he rewrote the entire program, u$1ng COBOL .computer lanotfage 






we could enter alT our elementary students in at least four j^o- 


\ 




gra.ms.with over-night turn-around if te§ts were careful lyscheduled. 
• * At about this t;imQ dissatisfaction^ with slow- tunir-a round, errors, 
lack of reading program had caused principals and/te;pm leaders to 
be less confident of the programs future. By yoow, though, the 




' 1 


Superintendent had become an ally of the pcogram. He called a 






meeting attended by allj interested partes in which the ^following 


\ 




agreements were made: / 






♦ 1. Rewrite program in Cobol / 
2. Try reading in 'grades*/^, S.and 6 In two schools * 
^ as sooh as possible/ , , 




•% 

' I * 


3. Set a schedule /f(/r spreading program 'to all fourteen 

elementary schools in 'an orderly fashion'as long a.s j?? 




1 » 


Improved st/dent performance Justified the effort. 
Dr. Harrison /and his crew were finally ready to put up* the 

, banks for readipig monitoring in early Spring. Two principals who 

/ 

hsd served on thfe BEST reading committee were ahead with instal- 






lation I'o* their schools. The reading consultants became the team ^ 




: ' / , 


leaders. Again we used many of the^strategies which seemed to Work 

V 

in Math.. We conducted no In-service i^iorkshops.* Ve allowed t;he 
teachers- to learn to use the sy^.em**by using the systfeirt. A pre--^. 




/ /' 




/ • 
/ / ■ 




/ / 


'llminary evaluation after only a short trial was very posltlv^e 




> / 
/ / 


dbout the effect&^^f BEST on student achievement. , 
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We concluded tfie^ year on a very cheery note compared to the 
feelings of the preceding surtiner. The teams were again -or- 
ganized to rewrite and re'fine the program. «One major change was 
needed though; we could not expand the program with the program 

installed by Technovat Jons* Mr. RQt4i&^nov^/. proceeded to write 

« 

and. Install his own COBOL version of the program. ThU work went 
on^ .al r summer. , \ ■ 

In the meantime, the reading team of tea\:hers and admln- 
Istrators; were Writing manuals (see Apperfd^x. K) i guides for parents 
and student, cross reference guides (see Appendix b) for materials, 
reading objective bank (see Appenijix M) , ^ test Items bank, (see 
Appendf'x N), and tests (see Append 1x 0) . 

In rewriting the computer program,. Mr. Rotol9)develop^ed 
another refinement. The cgmputer would now print all tests 019 
dittos (see Appendix O) for distribution to the schools* This 
v;ould now take us out of the priating business- — which was a'^ 
rather large part of the effort. High school girlst and ^other ' 
temporary help were recruited to do all the coding for entering the 
mil lions, of bits of information for both the reading and m^ath 

programs. ^ . 

4. , ^^^^ ^ *• 

We began year three of the program very enthusiastically. 

There were slight delays in testing as final computer adjustments^* 

\ ■ ' • ♦ * ' 

were made. 

Rfght from the Ibeg inning the new^ogram worked with an ef- 



100 



II- 



If 



■S2- 



flclency that was-a marvel" for everyone who Was"* involved. Tests 
were scored and reports produced. overnight . Test answer sheets ^ 
collected* by ^two o'clock In the afternoon brought all reports back 
to each school by nine o'clock the next morning. 

Another feature that Mr. Rotolo built into, thre program , solved 
90% of oury quality control problems. It was the printed answer 
sheet (see Appendix P) . The computer printed an answer sheet for 
every student with all Information „(names , numbers , sect Ions, - etc . ) 
except the answers to the test i^^s. The only writing now done by 
students was the simple mark sense fill-in. The Brentwood Op Sdan 
also permitted the use of regular #2 pencils -^solving another 
tpst problem. . , . /* 

The program went sotsmoothly that by ra.id';^ear the decision was 
made to add two schools to the-math testing progtam and in addition 
to add the reading testing program in the original BEST math 'schools. 
Studjes covering student progress in each of th6 6ubject were Iq ' 
proane^s. The biggest success of the program was now readily evident 
a$ parents-came to npeetings to learn How to interpret the test rer 



$i4lts that their children w^re now regularly bringing hone. 



, i / In the Spring I participated* In, a statewide workshop in Albany 
'to describe CAM and interest other districts. Brentwood had sufcceeded 
' Irt developing a full computerized -monitoring system in' math with 

state coop6ra\ion. Beyond* ^hat , on its ov/n, iSrentwood ha4 developed ^ 
* afeadinq monitoring syst^rn. At this writing in*1975t Brentwood is 

still the only district in New York State^with^a full ponitoring 
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program, completely computerized In reading. The Curriculum 
Coordinator Sw^embers of the Brentwood staff have now entertained 
hundreds of visitors. We ha^/ve gone to many other school- districts to 

'is ' ' * 

help them implement math programs. We have provided our objectives 
and test Items to hjelp people get started in reading as well. 

The hypothesis that this-program would help students achieve 
better results in math and reading has been confirmed, (See .ev- , ' 
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aluatlon section) . . / 


> 


\ 


An added bonus has been the affect development brought about 
by focusing on Instruction, Letting everyone know what we arel. 






trying to do and l€;,ttlng everyone know how he individually is 


l \ 




succeeding seems to ^erve as an Important motivational device* 




1 


As I onceVemarked to a teachers group, no one has ever met 






,a person who gets up in the morning and says to himself, 'Today 






I'm going to do^a rotten job/* Bi/t for some of us it v/orks- out 




t 


that way because we don't have enough Information about-the kind 






of job we are doing. Providing information, or feedback as ouy^ 






computer ^special ists choose to call it, makes possible the regular 
course correction for both teacher and student, 

". ' ■ ■ ' ' ' > ■ - 
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IX, Evaluatlbn ; To quote Dr, Daniel Stufflebeam, ''Evaluation Is 
In fact a processt^hFch can /provide key Inputs to aid planners and 
decision makers In making necessary decisions. Evaluation Inputs 
can provide these data not just at the end of a project or activity 
but throughout all stages, from initial 'drawing board' stages r 
through operation or implementation to termination or long range 
continuation.'*^ 

This definition has probably never been more applicable to a 
project than It is to, our experience with BEST CAM - (BEST is 
the Brentwood variation of Comprehensive Achievement Monitoring) 
Context * Evaluat Ion 

Q. Why did we need a monitoring system? 

A. Because student performance' ha^K^hown a regular declined 
Q. How much could vye afford? 

A. Little, (see Budget) We financed through ESEA. 
Q. How rruch couU we commit to long term expense for maintaining 

the program? 

A. Little, (see Budget) ^ 
Q. What experience did we bring to the project? 



A. Extensive work in currlcul um development . (see Implementations) 



Stuf f^lebeam, Daniel, 'The Process of Evaluation An.tasy Example" 
Unpublished handout distributed at Cluster meeting In Education, 
undated. 
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Q. What was the monitoring system that was needecj? 

A, One that would perform plerlcal task and provide 

, ♦ 
> usable Information. 7 CAM only one available which 

» ^ 

met the criteria. (see Problem) 

Input Evaluation 

Q. How long did we have to find a solution to d^clihlng scores 
In reading and math? - ' ' 

A. We needed a solution as quickly as possible. The downward 
trend was threatening to become a collapse. 
Project Plan and Callforn-la Test Bureau's diagnostic math and ! 
several other systems were compared. None met the crl.terla set 
In the Context Evaluation. .The Input and the context evaluations 
both suggest a rapid move, 
Proces s Eva 1 ua 1 1 on 

From the very beginning BEST was an outstanding example of 
an Innovation which produced' Its own regular process evaluation 
for indtyldtll^Is. and groups (see Program). It even produced Item 
analysU (see fig. 2k) and error analysis (see fig. 25)., By 
definition this monitoring program Is Process Evaluation. 
Product Evaluation 

In *0ctob^ri>^4 I testified before a Panel of the New York 

State Office of Performa^nce Review (Rockefeller's Klepak Commrsslpn 

^ - — — ■ ■ 

watchdog pn education). The Commission task was to evaluate test- 
ing procedures In- New York State. In nrvy presentatron,' I mgde 3s 
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strong a case as I could fbr the use of Criterion measures to 
Judge student success. We had amassed Impressive statistics In 
Brentwood to support this point of view. The most appropriate 
product evaluation Is the following statement that I made on the 
behalf of the Brentwood Schools at opening hearings held tjy the 
Office of Education Performance Review for the New York State 
Legislature. Th^^stat;lstlcs were compiled by Mr, David Holt, 
Administrative Assistant for Evaluation. ' , 
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SjATE OF New York 
Executive Chamber 
Office of Education Performance Review 
State Capitol 
Albany 12224 



September 24, 1974 



Dear Dr. Fournier: 

' ' * ' • • 

Thank you for agreeing to participate in the public 
meetings on pupil testing sponsored by this Office on Octo- 
ber 3 and 4, 1974 in Albany. The meetings will begin each 
day at 9: 3D A.M. because p€ Che large number of individuals 
speaking. 

We have scheduled your presentation for October 3 
bet^^een y:30 P.M. and 4:30 P.M. Following your formal 
statement the panel members may want to discuss some of the 
points you raise. Please bring eight copies of your state- 
ment wityy^. Enclosed is the schedule of speakers. 
Observers are welcome po attend. 

The meetings will be held in Senate Hearing Rdoni A - 
in the Legislative Of f ice. Building (corner of State and^ , 
South ^wan Streets). Enclosed is a map showing the loca- , 
tion of the. building. Please use the State Street entrance. 
One of our people will be in the foyer to greet you. 

If you have. any questions, don't hesitate to call me 
at (5I8) 474-3342 or 474-3170. We are looking fon^^ard to 
your participation. 



Sincerely, 




Dr. Raymond Fournier 
Director of Curriculum 
BrenD'^ood Public Sclpdols 
Third Avenue and Fourth Streets 
Br'enD^ood, IJew Vpr'-- 11717 

' ■ , . . 308 
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BRENTWOOD, NEW YORK 11717 
(5.16} 435-2123 



Daniel KI epak, Di rSc+cr 

Office of Education FerfonmanceRevIew 

Albany, I'.'ew York 



REPORT ON THE IMPACT OF ERENTWOOD'S " 

r 

CRITERION REFERENCED TeViNG PROGRAM 



Submitted by: 

Raymond Fournler 
Curriculum Coo^d.inatpr 



G. Guy diPIetro 
Arthur R. Brleg^r 
Raymond Fournler 
David Holt 



Superintendent 
Assistant Superintendent 
Curriculum Coordinator 



Administrator for Evaluation 



* ' ' ' i * * * 

In 197 1 i t'he Brentwood Schools began a trial of Comprehensive ^ . 

Achlevenent Ktonltorlng -In cooperation w|th^r.; Robert 0»Rel I ly and the 
Bureau of Research and'Cultbral Affairs. Comprehensive Achievemfent Monfto.rln 
most sf imply doscrlbed. Is a system foy . . 

a: specifying tWl^arnjng expected from the student during a semester. 

b. testing the students six times during the semester^ 

c. '{> reporting to [Jarents, students," teachers and administrators the 

progress that'js being made and the degree In wWch the learning 
being retained. Using the computer to score tests and compile re- 
ports made it possible to handle the mass of information. 
. There- wer^ many reasons for this move. First, and most compel I Irig^is 
that despite curriculum reforms, a variety of supervision strategies^ 
material Si changes, and sjljudent groupings, children in, our schools, were not 
making satisfactory scores on ' norm-referenced tests. We were In a situation 
in which we didn't seem capable of changing student performance on norm-, 
referenced, tests, yet" we believed that we could teach children to read and 
-perform "in mathematics. Our experience with- norm-referenced tests was con- 
firmed by .simi lar. frustration in school districts like ours throughout the 

country. ■ . ' ^ " 

The Brentwood Schools have more than 21,000 students in grades K-12. 
There are twelve (12) elementary schools, a 7th grade center, four (4) junic 
high schools and one'd) high school. StujJ^s In grades 7, 10, I I , and 12 
. are on double session. ' . - ' ^ 

* ■ Brentwood's assessed valuation per. pupi l Is the lowest Jn„ Its area. 
Even with a high rrfte of state *ald, Brentwood spends less per pupil than? 
neighboring districts.. The %tudent population is 77.9? White, 5.8? Black 
and 15.8? Spanish surnaned. ' ' 



i 
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^ . '^Target Schools . 14. 5^ aid to dependent ch 1 1 dren * ^ 

10. 5^ Black .... . . 

' • . ^* ■ ; • 

28. 8? Spanish suri»amed . • - . 

Criterion rfeferenced testing Is direct and straightforward. The task ^ 

I ♦ 
J) . ' ' ' 

and. performance expected are specified. The following example is from our 
6th grade program: V. . 

06-05-^*004-00 Finding averages of^vhole numbers ^ 



OBJECTIVE: Students will select the correct average 
number of any given set-up to five whoJe 
numbers. 

EXAMPLE: A student received 'on five tests the following 
grades: 70,90,80,85 and 100. What was^his 
^ average grade? 

■(A> 90 . ^ (B) 80 

(C) 70 • • (D) 85 

* . . ^ First Semester 

If the task is appropriate, success is achieved when the student solves 
problems of this 4<ind. Being above or below the 50th percentile in some part 
of the California Achievement Tests is rather Irrevelant to the teaching and 
learning task Involved in this mathematics objective. 

starting In 1971 specified objectives and tests In Mathematics for 
graiJes 4, 5, 6. in 1972 we developed objectives and tests for reading at' the 
elementary level. We distributed the dbjectlves and sample test Items, to 
students and parents. Computer reports on student perf orliiance were provided.: 
to teachers,. students and parents. 

5.tudents worked toward t*he objectives, and criterion tests lndica;te^ that 
they were succeeding. Performance Improved- from test to'-test in botti math and' 
reading. Teachers, parents and principals reported satisfaction that students 
were learning. Mater iaT?^a;id activit-les were being organized to help sluden+s- 
achieve the objectives. , 

. ■ . ^ ' - 111- ^' ' 
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StI 1 1 the questions were heard, "How wi 1 1 they <io on normyreferenCed * 
tests?", "How-w.ni they compare to the- norm?" ^ , , • • . 

it is somewhat a surprise that students work! qg In' erst terlon referenced 
InstructlonS'^ave regularly Improved thelr^ perfohmance on normrref erenced 
tests since 1971 : It places us In the position of using honn-ref erenced . 
testing as validation for criterion referenced instruction. ' the dilemma 
occurs because criterion referenced Instruction defines success ^s the per- 
formance of the task while norms define success as a position relative to 
other people'N/ho took the testl 

'While using the data from nQrni-referenced tests to demonstrate success 
of our criterion re\renced instruitlon, we are uncomforfabl^ because if may 
be Implicit recognition of . the power and acceptance enjoyed by +he, norm- 
referenced tests. Yet, the fundamental flaw of norm-referenced tests Is.that 
only one-hall of the population can succeed by being above the 50th percentl le 
• One-half must -fall, regardless of, the. Instruction: Regardless of the tasks 
that students can perfo'rm, one-half the population must t,e below the 50th 

. ' 

percentl le on these tests. " . " 
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CHART t Is a comparison of mean scores of children In thd three ESEA 
tsrget schools having a 'studeoj population that Is "most sBrlou§^y dls- 
advantaged to the. mean scores of the other three target schools* The 
test used In the cornparlson Is thevmath computation subtest from the' 
CalHornIa Achievement Test, Level 3, Form A, 1970 edition. The compjete 
irath and reading tests were given to the children In ail elementary 
schools In grades A, 3 and 6.„as part of the DIstrlctwIde testing program 
during May of 1971 efnd 1972 and February 197?. The graphs reveal that 
although the CAM group was significantly below the non-CAM group In May 
of 1971 at the end of grade four preceding the Introduction of CAM math, 
by May of 1972 the CAM group h'ad pul led abreast of the non-CAM group at 
the, end of. grade five. As sixth graders, the CAM^group had pulled away 
from the non-CAM group, and although the difference was not quite sig- 
nificant in February, a significant difference was projected for the end 
of the y^ar. - ' , - ^ ^ , 
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CHART 2 Is a comp3rIson of a sample of students 'from a control and an 
e>iperlmental group, In one, school that Introduced CAM In reading In March . 
of 1975. The Gates-MacGtnltle Reading Test Leyel D, Formal was given as 
a pretest at t'he end of January/ and Form 2 was given as a posttest at 
the beginning of Jjiine. Although the CAM was to begin the flr$t week In . 
February, several problems delayed the start Until the last week of March, 
J Only ^orty-two ^Instruct fona I days elapsed' between the first CAM test and 
i^fhe last CAM test. After • Inltia I differences between the two groups due 

to chance variation were equal I zed through ana lysis of covarlance, the 
* posttest means were compared. As shov/n In the graph, the experimental 
group Improved their reading comprehension more than the control group. 
When the adjusted pottest means were compared, the difference approached^ 
.statistical significance with a probabi I I-ty of, •OTS. This means that there 
Is less than eight chances In a hundred that' this difference could be due 
to random varlation^^lWthl Idren's test Scores. 
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CHnRTS 3 and -4 show the progress of 2 classes throu^ gfades 4, 5 and 6 
In their performance on the Call.fornia Ac)ileveraent T^st - Reading Compre- 
heoslon-and Math Computation. Chart 3 Is for Twin Fines Elementary where 
CAM was first Introduced In math and. Chart 4 .Is for Northeast Elementary 
wbere CAM was~flrst Introduced In reading. .Although no research design 
nor statistical tests were applied In these <iases, vqry Interesting spurts 
of growth are noted for the children participating In the CAM system 
relative to when the systiem was Introduced. • 
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CKtART 5 (handout) Is a comparison of the mean scores from the sixth grade 



NYS PEP Math Tost between two groups. One group Js comprised of children 
In the three ES^ target schools that first received CAM.. The other group 
Is comprised* of the children In the other eleven schools In the District 




which-have nat used QjAM^urlng the t-lme of the comparison. The progress 

of therCAM school s counters the New Yor«k State downward trend In sixth 

\ 

grade mathematics scores as reported on page II of the Statewide Report of 
October 1973 Reading anid MatTielnatlcs Test Results , The non-CAM school s 
mean math scores .are also-following a pattern similar to that of the state 
as a whole. Trends such ^s these provide substantial evidenpe that the 
CAM system In Brentwood s's an effective tool for use In helping children 
acquire mathematics skills. 
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BRENTUOOD. NEH YORK 



il.50 

31.25 

31.00 

30.75 

30.50 

30.25 

30.00 

29.75 

29.50 

29.25 

29.00 
28,75 
28.50 
28.25 
28.00 
27.75 1 
27.50 
27.25 
27.00 
26.75 



26.50 



Non-C.A.M. Schools 



C.A.M. Schools 



. 10/73 



IoTtT 



10/72 



FIGURE 1: ' . ' 

Mean Score Comparisons on Three Administrations of the Sixth. Grade N.Y.S. P.EJP. 
Math Test of Three E.S.E.A. Target Schools using C.A.M. in Math to the other Eleven 
Schools in Brentwood not u§^g the C.A.M. system. 
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.er|c 



An Information system that gives Regular/ reports on achreven\ent and 
retention, he^lps. both the learner and the'teacft^r. -^The CAM* (nformatldn 
system ''does not prescribe materials or methods 'txut requ I res^ frequent 
teacher decisions concerning strategy and tactics .to help" students achJeve. 

Brentwood's three year results on a variety of^testg; support the Idea 
that specifying ta'sks and testing (Jerformance .wl I I Improve the student's 
performance In reading and math for? grades 4, 5, and 6*. ^ • . 

Observing students behavior on these specJfled tasks seems more apn 
propr|ate than measuring against a norrrt which sorts all learners In-tp . 
above and below percen-Mles or stanlnes. * ^ ' 
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. .4 












'.J 






. -Ill,- , 

» 




\ 


The Product. Evaluation submitted to the Klepak Commission used 




f i 


norm-referenced tests for comparison. If we really believe 1n ^ 
criterion referenced tests, the analysis of objectives and per- 






V formance used In evaluating a program should measure student 

crcwth against the original criterion. Mr, Dav^ Holt has compiled 






the following report on our student's progress toward the set of 




• 


» 

objectives for each level • Overwhelmingly there is*positi,ve growth. 






Where there is not, the math committee will study the situation to ^ 




- ■ . 


bring about ode of the following: 






1 . better student perforjmance 






' * * . 
1. change in the objectives^ 






3. change in test items ^ • 

* 






k-. change In method or material for instruction/ 




ft 


The evaluation of the program to this point has encouraged 
the decision to monitor reading and mathematics in .the junior high 






schools as well, begiipning In i^5*76. 

t 


J 


0 


a - • . ■ 

/ . 

12x3. • 
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BRENTWOOD UMION FREE SCHOOL DISTRICT * 
BRENTWOOD, NEW YORK 

• '"* ■.'' \ , • . . • 

TO: Arthur R* 'Brleger 

FROM: David S. HoU 

RE: BEST HATH ANALYSIS OF J)BJECTIV€ PROGRESS DURING THH 1973-197* 
SCHOOL YEAR . • 

DATE: April 9. 1975 * • 



At the request of Mr. Fournler, I conducted an analysis of the BEST Hath 
progress during the 1973"?^ school yedr, ^ 

Attached are several ' tables that examine the progress of children Iti 
achieving success on each objective within each math level. This is the 
first co:rprehensive view of math achievement shown in terms^of the 
criterion referenced test scores for a full yeal" period. The results of 
the analysis are very interesting; they show important positive changes; 
they should be very useful to the BEST Hath Revision Committee, ^ ^ 

The first table in each level is a frequency distriliution of the percent 
correct for each objective on the first test pre- instruct iona I and of the-^ 
percent correct foh each object! ve on the last test post-instructional. ^ 
This table shows the median percent corre^^for all objectives pre-ins ti:ujC- . 
tional, for all objectives post-instructional and for the vincre'ase fn median 
percent correct; It also shows the range of percent correct for the 
objectives bn the first test and on the last test. It will be noted that 
in all three levels the range harrows considerably as the median rpercent 
correct is raised.' these tvvo bits of information show that th^ pupil *s 
progress has the pattern expected when teachers utilize the procedure of 
teaching for achievement of behavioral^objecti ves and those objectives 
are measured with a criterion referenced test. 

* . - ■ ' 

The second table provided data on individual objectives. The fj rsX column 
lists In which semester the objective was taught ^d the second' column lists 
the percent correct of the Jtems for each objective on the first test prior 
to instruction* The next column indicates the percent correct of the Items 
for each objective on the last test of the semester, while the fburth column 
lists the increase in percent correct, i-e., column three minu^column two. 
Column five lists the percent correct of the items for the objectives taught 
•both s^mestersj and the next column lists the increase in percent correct. 
I.e., column five minus column two. The data inrxolumn four also hive plus 
signs {+) beside the increases that are above the median for the group of 
objectives. Those objectives above the median increase are verbally identi- 
fied in the last column. ^ . . ^ v 

The last table presents the frequency of increases in percent correct for 
all objectives for only one semester and for both semesters. 

a 

These data summaries will help the review committee make decisions about 
which objectives to keep and which objectives need revision. 1 would be 
glad to explain these tables in person if you felt it would be helpful. 



DSH:cb 

cc: Mr. DiPietro 
* Hr^FourrtTer 
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BREMTVOOO UNION FREE SCHOOL DISTRICT . . 
BRENTWOOD, MEW YORK / 



GRADE h > BEST HATH 



Frequency of X Right 
Pre- f ns t ruct i ona 1 







i. 


f. 


cf 








96-100 


- 












91- 


95 






• 








86- 


90 








111 - . 


3 


«2 


; 81- 


85 








nil 


ii 


39 


76- 


80 


1 \. ■ 


1 




111 


3 


35 • . ,-\ , ' 


71- 


75 








nil 


V 


32 


66- 


70 






0 


J4+1 hi 




28 1 ^ .' - - 
















(Hcdl^D - 66.1) 


61- 


65 


11 


•2 


41 


Am 


5 


20 V ' , 




60' 








mf 11 - 


7 


15 


















51- 


55 




5 


39^ 


nil 




8 




50 


111 


8 


3^^ 


111 


3 


h 


k\- 


h5 




11 


26 












(Median 




43.2) 








36- 


40 


1 


1 


15 


1 • 


1 


^\ 




35 


1111 


h 


. J'i - 








26- 


30 


111. 


3 


10 


01 f Terence between medians Is 


21- 


25 


Ill 


3 


7 








16- 


20 


nil 


k 


k 


* 4 






11- 


15 


/ 












6- 


10 














1- 


5 








• 







Frequency of X Rl^ght 



tlange 6*1 



12f5 
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BRENTCOOO UNION 'FREE SCljOOL DISTRICT 
BRERTOOOD, NEW YORK 



GRADE 4 - BEST MATH 



Objective 


Semester 

Taiioh t 


Test I 
S Right 
Pre-In- 
structional 


test 6 . , ' 
S Right , • 
Post-In- . 
structi«i*T . 


Increase 
.in « 
. Riaht 




1 

1 • 


46 


64 


18 

» 


413 


1 


34 


I 56^ 


22 


414 


2 


' 43 


59 


16 


415 ' 


1 


43 


56 


13 


/ 416. 


2 


25 


37 


12 


• 417 


2 


20 


53 , 


33 + 




o 


50 


69 


19 




• 1 


64 


83*^ 


19 


4ZZ 


1 " 






17 


423 


1 • 






' 37 + 


424 




4 


46 


27 + 


425 


2 


49 


66 


17 

28 + 


426 


, 2 


41 


69 


427 


2 


25 


57 


32 + 


431 


2 


26 


48 


22 


432 


2 


35 


68 . 


33 + 






20 


46 


26 + 


'434 , 


2 


45 


86 


■ 41 + 


435 


2 


55 




. 29 + 


436 


' 2 


34 


* 48 


14 


437. . 


1.2 


24 • 


55 , 


, 31 + 


451 


1.2 


43 


60 

^- . 12 G 


17 



T6st 12 
S Right 
Post^In- 
structional 



Increase 
In % 
Riaht : 



^ - 52 



^64 
71 



33 



40 
28 



Objective 
^Description 



CoTTori^, 
Factor^l 



Kultiplicetiot; 
Division/ 

Kultiplicatio 
Division 



Equivalent 
Fractions 

Kixed Numbers 

Add FractiQnS 
^w/= Denom. 

Subtract Frac 
w/= Denom. 



Equivalent 
Fractions ' 



BRENTOOOO UNIOM FREE SCHOOL DISTRICT . , 

' BRENTl^OOD, KE« YORK 

^ GRAPf'A - BEST HATH 

Test I Test 6 " Test 12 , 

5 Right ■ S Right . increase 2 Right ^Jf^*^® ns-to/ti;* ' 

Oh^orHve Se~sster Pre-In- Post-In- " in t Post-In- *n 5 Objective 

SSJiei TauSr st rcctional structional Rioht structionat . Richt DescriotTOfL 

36 Basic opera- 
tiorts w/nuniet 
sent 



452 


1,2 


41 


68 - 


27 + 


77 


453 




45. 

• 


■ r 82. 


■ 37 + 




461 




55 


* J 

69 


.14. 


73 


• 46? 


. 1 


• » 49 


60 


.11 




463 


1. 


52 


63 


.11 




464 


1.2 


■ 49 


,68 


19 ' 


81 


465 


^1.2 


• 43 . • 


71 


28 + 


72 


466 


2 »■ 


3T 


- 56 


25 + 




467. 
468 


1 

1.2 


48 
64 


61 

91 


13 • 


fin 


469 


2 


44 


64 


20 


* 


. 4610 


2 


27 


55- 


28 + 




471 


1 


19^ 


. 60 


41 + 




472 


1 


46, 


70 


g4 + 








50 


66" 


16 * 




474^ 


2 


42 


, 69 


• 27 + 




475 


2 


44 


70* 


" 25 + 




476 


2 . 


29 


55 


26 + 




482 


2 


54 


79 


V 25 + 




483 


2 




90 


14 





18 



Number sent 
w/Fractions 



•32 

^9 ^Divisiorf wofSf 
Prohlems 

+t - word pro 
v//fractioniS 



26 Reading grapji) 



Division v/ord 
problems 

English linea 
measure v// + 

Precision of 
lin^measureme 



Enqlish measi 
W/Add.& Subtt 

* \ ■ ' 

Time measure 

English linec 
measure w/ - 

Identify 3D 
figures 
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Increase tn 
X RlQht 



6-10 
1-5 
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BJfeNTCOOO mim FREE SCHOOL DISTRICT 
BRENTOOOD, NEW .YORK 

GRADE 4 - BEST t'ATH 



Objective Jreoiiency Distribution 



One Semester 

f cf 



56-60 








51-D9 


4 






46-50 








4T^45 


11 i 


2 


42 


46-40 . 


IT 


2 


40 


31-35 


HIT 


4 


38 


26-30 


4w -mr 1 


11 


34 


21-25 


aw 


5 


23 


16-20 


' 23.5 Median 


10 


18 


11-15 


111 


8 


. 8 



Both Semesters 
t f cf 



11 


2 


8 


11 


2 


6 


111 


•a 
»^ 


4 


r 


1 


1 



30.5 Median 
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BRENTVOOO UNION FREE SCHOOi; DISTRICT 
BRENTWDOO, NEW YORK 



' CRAPE 5 - BEST MATH 



96-100 

9J- 95 
86-90 
81- 85 
'TS- 80 
71- 75 
66- 76 
61- 65 
56- 60 . 
51- 55 
A6- 50 
hi' 

36- ko 

31- 35 
26- 30 

21- 25 
16- 20 
11- 15 
6- 10 
I- 5 



Frequency of « Right 
. Pre-lnistructional _ 

t f cf 



1 
1 

in 

Mil 

111 
I 

w+r 11 
111 
111 



I 
I 

3 

4 
3 
I 
5 
7 
3 
3 



58 
57 

56 
53 
kS 
46 
45 

uo 

33 
30 



{Median = 33.8) 
i^rt J-m- 13 27 
11 



J+H- II 
1111 

I. 
11 



7 

1 

2 



Iti 
7 
3 
2 



Frequency of % lilght 
Pos t-lnstructional 



t 




cf 


1 


1 


• 58 


11 \ 


2 


57 






55 


4-m 11 


7 


53 


mr 111 


8 


•46 


j+tr 1111 


9 
12 


38 
(Median 
29 




10 


17 




5 


7 


1 


1 


2 


1 


1 


• 

1 



Difference between raedians is 31.7 



Range »=.79 



Rangci » 59 
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^ . ' BRENTCOOO U?n ON FREE SCHOOL DISTRICT 

BREtoOO, HEW YORK . 

GRADE 5 - BEST HATH " ' " . 

- ■ Test 1 Teste Test 12 ' 

S Right S Right Increase S Right Increase 

Objective Semester Pre-ln- . Post-Ih- .in S . Post-In- in i Objective 

■ number Taught - structional structional Right structiona1_ Right Descrioticn 



511 


1 


68 


75 


• '£1 


513 


1 


60 


67 


7 


514 




25 


65 


40 + 


51^ 


2 


28 


41 .' 


13 


516^ 




30- 


51 


21 . 


517 . 


2 


38 


63 • 


25' 


518 


2 


43 


6« • 


25 


519 




61 


75 


. 14 


521 


i 


82 W 




4 


coo 


1 


/O 


oo 
oZ 


iZ 


523 


1.2 ^ 


46 


. 76 


30 + 


524 




35 


65 


30 + 


531 


• 

2 


36 

* 


63 


27 


532 


> 2 


33 


^ it- 

64 


• 31 + 


5^3 


- 2 


22 


60. 


38 + 


534 


2 


16 


55 


39 + 


535 


2 


17 


58 . 


. 41 + 


536 


2 


52 


74 


« 

22 


537 


2 


26 


63 

• • 


37 + 


538 


? 


28 


62 


34 + 



.130 



Rounding to 
1.000 • 



66' 20 . Multiply t.vo 

0 



3-digit nos. 

56 19 Division of 

5-^digi,tS by 2 
gits V// remainder 



Least cormon 
denominator 

Adding fract. 

w/unlike 

denominators 

Subtract free 

w/unlike 

denoiynators 

Multiply wholi 
# and-a fract 



Add mixed ? 

w/unlike 

denominators 

Subtr. mixed 
# w/unlike 
denominators 
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.1 



. Tire?'' 3si!-l£i^ MSiisBi 



noanc S - BEST KATH 

•fe^ 6 
"Right 



S39 
5310 

5311 



2 
2 

No 



39 
27 

18 



64 

57 

57 



541 


9 
C 


27 


80 


542 


<• 


42 


77 . 


543 


.2 

' » 


19 


57 


544 




63 


74 


545 


2 


. 16 


52 


551 


1 


25 


52 


552 


1 


. '28 


66 


553 


1 


23 


63 


554 


1 


"27 


72 


555 


1,2 


48 


75 


O556 ■ 




47 


67 


557 


2 


45 


74 


558 


2 


49 




561 


1.2 


33 




562 


1 


78 


89 


. 563 


1 


58 


74 


564 


1.2 


47 


66 



test 12 
Increase S Right 



in 5 
Right 

25 
30 + 

39 +■ 
53 + 
35 + 
38 + 

42 +' 
27 

40 + ■ 

^ 45 + 

27 
20 

.29 + 

19 
11 
11 
16 



Post-In- 
structional, 



Increase 
in S 
Rioht 



69 
65 



54 



21 
18 



21 



Objective 
Dpscriotlcn 



Subtr. mixed 
f w/onlike 
denominators 

Division of 
fractions 

Changing frac. 
Xo decimals 

Changing dec* 
to fractions 

» Adding decina 



Piece value 
fpr decirals 



Associ ative 
proo. of 
moltiplicatic^ 

Conrputative 
of 

muUi plica tic 



Hissing frac; 
§ facts in §, 
sentences 



70 



23 



ERIC 



BREJIU'OOD U.'JION FREE SCHOOl DISTRICT 
GJiAOHL 5 - BEST MATH 



• • Test I • Test 6 

S Right S Right 

r Objective Sefester Prc-In- Post-In- 

Kirsber Taucht structicnal struetionaT 



565 
566 



1*2 
2 



43 
22 



73 
59 



Test 12 
Increase S Right 
.in S Post-In- 
Rloht structionaT 



20 
37 + 



64 



567 
568 
569 
571 



1.2 
1 

1.2 
1 



41 
63 
26 
27 



60 
80 
71 
56 



19 • 
17 

45 + 
29 + 



70 



60 



572 
575 



1.2 
1 



60 
27 



83 
60 



,23 
33 + 



85 



576 



27 



61 



34 + 



577 



30 



72 



42 + 



578 
581. 

582 



1 
1 

1.2 



66 
-63 

n 



79 
92 

62 



13 

29 + 
51 + 



64 



r -583 
585 

586 



2 
' 1 

. 1 



9 

,41 
24 



36 
77 



.9 

37 + 
53 + 



587 

588 



1 

2 



21 
41 



76 
69 



5.5 + 
28' 
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BREHTOOOD UNION mt SCHOOL biSTRICT 
BRENTOOOD, NEW YORK 



A 



GRADE 5 - BEST MATH 



Increase 



in 



One Sccester 



Objective Frequency Dist'ribution 



56-60. 

46-50 
41-45 
35-40 
31-35 
26-30 
21-2& 
16-20 
11-15 
6-10 
1- 5 



nil 



4W- 
Wt 1111 

.Median 27.5 
4W 1 

4w n . 
aw m 
111 
1 



5 

9^ 
5 
10 
6 
7. 
8 
3 
1 



cf 

58 

54 

AO 
35 
25 
19 
12 
4 
1 



Both Semesters . 
t. f cf 

1 11 



1 
1 

111 



1 
1 

5 
3 



10 
9 

8 ' 



23.0 Median 
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BRENTWOOD UNION FREE SCHOOU DISTRICT 
BRENTWOOD, NEW YORK 

GRADE 6 - BEST HATH 



Frequency of.?; Right 
P re* I ris t ruct t ona 1 



• 




J 
L 


Cf 


< — * 


X 


cf 


9o-i00 














91- 95 








1 . . 


1 




86- 90 


1 


< 1 




Ill 


3 


53 


Oi Or 


1 


1 


53 


111 


3 


50 


76- 80 


11 


2 


52 


1111 




ff 

47 


71- 75 








4+n' 1 


6 


'»3 


66-. 70 


111 


3 


50 


J-Hi 111 


8 


37 


61- 65 

56- 60 


% 

11 


2 


hi 


mt nil 

j+H" J-H1 11 


9 
12 


29 
20 


51- 55 


nil' ^ 




15 


nil 


4 


8 


46- 50 


1111 




ff « 

41 


111 • 


3 


k 


Hi- h5 


111 


3 


37 




1 


. 1 


36- ho 


J-HT 1 


6 


3'» 








3t- 35 
26- 30 


J4-H 1 


6 

(Median = 
n 12 


28 
22 








21- 25 




5 


10 


\ Difference betweien 


16- 20 


1111 


ii 


5 








11- 15 


1 


1 


1 








6- 10 














1- 5 















Range - 79 



Frequency of ? Right 
Post- Instructional 



(Median « 64.4) 



is 30 



Range « 50 
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BRENTWOOD mm FREE SCHOOL DISTRICT 
BRENTWOOD, MEW YORK 

GRADE 6 - BEST ^TH 



Test 1 . Test 6 lest 12 , . 

) • S Right * Right Increase - IS Right Increase 

Objectt««r Semester Pre-In- Post-In- in 5, Post-In- in S OWective' 

Numbgr Taught ■ sllructional structional Richti • structiogal -Rioht, Description 



fill 
oil 




24 


59 


35 + 




613 


1 . 


18 ' 


" 45,- ■ ■■ ■ 


27 . 




614 


2 


25 


61 






615 


2 


29 


59 


30 + 




blo 


1 o 




70 


97 
cc 




oj/ 


1 o 






1 / 


68 


618 


.1 


' 35 ' 


62 


27 




619 


-.2 ' 




. 59 °" 


18 




€Z\' 


1 




. -91 . 


5 . 




622' 


1 ■ 


-84 


88 


4 




623 


1 


68 


78 


10 




624 


1.2 • 


47 


63 


16 


62 


631 


1.2 . 


36 . 


58 


22 


58 


d3Z 








3? + 


62 


633 




> 28 


69 


41 + 


63 


634 


T.2 


. 19 


61. 


42 + 


59 














635 


1.2 


30 


71 


41 + 


65 


636 


1.2 . 


' 31 


, '57 • 


26 


55 


637 


1.2 


22 


' 62' . 


40 + 


53 



22 
15 



Scienti fic 
rfptation 



pounding to 
nearest decimal 
fractioji ' 

Decimal In- 
equal itiesr 



15 
22 

35 + Fractions w/ 
unlike denomina- 
tors » 

35 - Eractfons w/ 
unlike denomina- 
tors 

40 Multiplication 
of Fractions 

35 Division of 
Fractions 

24 

31 - Fraction^ w/ 
unlike denomina 
tors 
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• BRENTWOOD UNIOM FREE SCHOOL DISTRICT 

BRENTWOOD, NEW YORK 

* . GRADE 6 - BEST ^ATH 

Test I Test 6 Test 12 

• • S Right S Right Increase J Right Increase 

_ ,2 47 75 -28 + 72 25 . HuHipT^- cation 

638 1,^ , effractions 

Division of 
Fractions . 

Place value for- 
decimals throach 
lO^OOOths 

Reading & v;ritint 
decimals through 
lO.OOOths 

Changing Fractioi 
to decinals 



Division of 
decimals through 
hundredths 





639 


1.2 


42 


70 


28 + 


70 28 


* 


641 


* i 

"••2 


20 


7? 


52 + 

t 


66 46 




642 


' 1.2 


46 


79 




72 26 


• 

• 


• 

643 


2 


30 


58 


CO ▼ 






644 


2 


29 




1 7 




• * 


645 
646 


2 
2 


77 
40 


84 
60 


7 

20 


• 

> 




647 


2 


28 


60 


• 32 + 






648 


2 • 


27 


52 


25 


• 




651 


2 


23 


67 


44 + 






652 


2 


29 


59 


30 + 






653 


2 


0 

19 


60 


41 + 


t 




. 654 


1 


. •. 49 


•77 


28 + 






651 


1 


0 

55 


71 


16 






662 


1.2 


28 


36 


• 8 


50 22 




663 


' 2 


43 


58 ' 


15 






•. 664 


1 


• 


71 


15 




. 1 






1 

* 


• 

130 







Changing ratio 
to perc*<*nt 

Finding Percent 
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X. Budget . . . . - 

A. Financial Budget . ~ 

'1. Start-up Cost - Tst year 

a. 25% per student charge by Grumman $ 275-00 
Data System for 1100 students. 

ESEA funded. •/ 

b. $500.00 per team leader for extra ^ 1500.00 
ser*vices« ' . 

c. Three p/lnclpals X 3 weeks salary 5^00.00 
for workshop and writing. 

d. Three teachers X 3 weeks salary for^/"^"^ 3600.00 
workshop and writing' . 

Total Start-up Costs $10,775.00 

2. Current Costs 

a. 9i per student per test processed by 
Brentwood Computer. 

•A 

b. Twelve Math team leaders at $500.00. 
, c. Net cost to district for writing 

(summer '75) Junior High objectives \ 

In math and rewriting elementary math ^ ^ 

and reading. Total Cost Is $20,000.00 x 

but through BOCES Cooperative Aid / 

Project, 80% (dFstrlct aid ratio) Is ' , 

reimbursable. ' . 



4-131- \ 
d.* Twelve Aides X 3 hours per weekv 
B. Time Gommi tment ^ 

The Curriculu^ Gbordinator has glyen up to kO% of his time in 
some periods to the management of this project. . 

' It has involved all teachers and administrators inestimably 
from a time point .of view. 
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XI. Self Assessment . Luck Jiad a great deal to do with the 
successful implementat^^of thii program. I woul^^ke to believe 
that hard work had even more to do with It. I ha^^BM^iiresponsible 
for the deve^lopment of educational progr^ams In the ^gjjpn' district 
since 1963, Many of our programs have achieved national distrl- 
butlon. BEST however, was the most complicated, needing more 
matefl^als, people and strategy than any other. • 

Its scope Is overwhelming in some regards. It comes at a 
time 'When bel^avsoral objectives and accountability are contrpver- 
slal. It has done what It was supposed to do and yet It will never 
be finished. It cohtipjues to offer great hope for the organization 
of more ef fective^^Ynstrucrt lonal programs. Tt Is modern In Its usb 
of technology anc^^^up-to-^ management systems. Yet the potential 
for mis-use is signi f ic^^n^. It Inust be watched because If It Is 
jnls-used, at best it would fail and disappear;' at worst it has 
the potential to be used for oppresslorj of students and faculty. 
This Is probably true of all effective measures. Managers are 
usually caught In between doing nothrng and doing some very pos- 
Itlve things which have potential for mis-use. Just as we are not 
likely to give up books, radio, newspapers and television because 
they have the^potent ial to be^Is-used, we will not pass nip thFs 
fine tool for that ^amie reason. This becomes just one more factor 
that Yequi res professional vigilance. 

Assessing rty own performance In the progran, I can't think of 
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much that I would change. If I were doing It again, I would 
like to have more money and a larger staff. ^ 

We once had a group of/^isltors from one of the school (TTstricts 
In New York City, (roughly the size of Brentwood) They were ex- 
pending $300,000 In an attempf'^to implement this system^ The 
prpject coordinator worked with the project as his one and only job. 
With a tiny traction of that budget, with this a small part of , my 
responsibll Ity, there^were times when It seemed veny- djfjf Icul't . 

y 

Since the program Is a success I wouldn't change any decision- 
for fear that It would precipitate the development of some uncon- 
trol lab }^e, situation. • . ' ^ 
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